I'm using manage.py test --parallel to run my tests and want to create a separate log file for each test runner.
Currently, all the test runner worker write to the same log file, so I get a single log file with contents "striped" like this:
[ForkPoolWorker-2] ...
[ForkPoolWorker-1] ...
[ForkPoolWorker-1] ...
[ForkPoolWorker-3] ...
[ForkPoolWorker-2] ...
[ForkPoolWorker-4] ...
[ForkPoolWorker-1] ...
I create a custom configure_logging() method like this:
import logging.config
import os
def configure_logging() -> None:
"""Custom logging configuration with process-id named files"""
process_id = os.getpid()
log_file_path = f"runner-{process_id}.log"
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"file": {
"level": "DEBUG",
"class": "logging.FileHandler",
"filename": log_file_path,
},
},
"loggers": {
"django.log": {
"handlers": ["file"],
"level": "DEBUG",
"propagate": True,
},
},
}
logging.config.dictConfig(LOGGING)
I'm using a custom test-runner to wire things up:
from django.test.runner import DiscoverRunner
from myapp.tests.logging_config import configure_logging
class ParallelTestRunner(DiscoverRunner):
"""Parallel test runner with separate log files for each worker"""
def setup_test_environment(self, **kwargs):
"""Configure the test environment with our custom log setup"""
super().setup_test_environment(**kwargs)
# Configure logging with a unique file per test process
configure_logging()
I wire that up in settings.py like this:
TEST_RUNNER = "myapp.tests.runners.ParallelTestRunner"
However, when I look for the log files, it appears it's only creating a single log file like runner-16.log, so I think the name-generation for the log files it happening in the testrunner main process, before the worker split.
Any ideas on how I can wire this up so that the log files get created per testrunner worker? I'd like to see log files like:
runner-11.log
runner-12.log
runner-13.log
runner-14.log
runner-15.log
runner-16.log
Thanks!