Why is python structlog logging a redactError every time I add a list as a key word arg?

Question

I'm using python structlog version 19.1.0 (https://www.structlog.org/en/19.1.0/index.html) and in a previous deployment, I used this same version and did not get any redactError messages in my logs. But now, whenever I add key-word args to a log message (especially if the value is a list), it also logs a redactError.

My code:

logger = structlog.get_logger()
logger.debug("My custom event name", email_addresses=["[email protected]"])

The log message:

{"event":"My custom event name", "email_addresses":["[email protected]"], "level":"info", "msg":"\u003cnil\u003e", "redactError":"1 error occurred:\n\t* error redacting item email_addresses: 1 error occurred:\n\t* error redacting item 0: unknown interface interface {} not redacted\n\n\n\n","request":"b8214486-d270-416b-ba3a-cc36b784c19d", "stream":"stderr", "thread":140500243846912, "time":"2020-07-22T19:28:50Z","timestamp":"2020-07-22T19:28:50.737388Z"}

I have no idea why it's giving me a redactError when it is clearly still able to add the key word arguments in the log message. I've also noticed that if the list is longer than 1, it will add more errors to the redactError string (e.g. "error redacting item <>: unknown interface interface {} not redacted\n" for every single index).

I'm really baffled as to why I'm getting this error. It hasn't interfered with my code's ability to execute, it just really clutters up the logs. Any help is appreciated!

Edit: thanks for some of the comments below. Adding my logging configuration in case that is the issue:

# setting up the logging config
level = getattr(logging, "DEBUG")
formatter = logging.Formatter("%(message)s")
sh = logging.StreamHandler()
sh.setFormatter(formatter)

logger = logging.getLogger()
logger.setLevel(level)
logger.addHandler(sh)

processors = [
    structlog.stdlib.filter_by_level,
    structlog.stdlib.add_log_level,
    structlog.stdlib.add_logger_name,
    structlog.processors.format_exc_info,
    structlog.processors.TimeStamper(fmt='iso', utc=True),
    structlog.processors.StackInfoRenderer(),
    add_thread,
    structlog.processors.JSONRenderer(),
]

structlog.configure(
    processors=processors,
    context_class=structlog.threadlocal.wrap_dict(dict),
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

Answer 1

It appears the issue is structlog.processors.JSONRenderer.... When I change structlog.processors.JSONRenderer() to structlog.processors.KeyValueRenderer(), the redactError doesn't happen. I don't know why JSONRenderer doesn't like lists, but maybe someone else knows.

Answer 2

This error does not come from structlog.

It looks like you have either a structlog processor or a stdlib logging handler that is trying to redact personal identifying information from your log entries and it seems to be confused by the fact that you email addresses field is a list and not a string.

We'd need more information about your logging and structlog configuration to tell more.