Pytest coverage with django generates wrong result when run in parallel

977 Views Asked by At

When I run my testsuite with parallel executions I receive different results compared to run only one test runner.

I tried two approaches and both did not provide proper results. These commands did either only produce one partial result or less than we had runners.

I've tried with coverage and pytest combined:

COVERAGE_PROCESS_START=./my_app coverage --parallel-mode --concurrency=multiprocessing  run -m pytest -m "not e2e" -n 4

Also with pytest and pytest-cov:

pytest -m "not e2e" -n 4 --cov=my_app

The second one also had the issue that some templatetags were not seen as registered even though others in the same directory were registered.

After running these I executed coverage combine and coverage report. When run in parallel the results are always incomplete compared to running it with only one test runner, which works perfectly fine:

coverage run -m pytest -m "not e2e"

This is my coveragerc:

[run]
include = my_app/*
omit = *migrations*, *tests*
plugins =
    django_coverage_plugin

[report]
show_missing = true

Does some know how to get proper coverage results when running pytest in parallel?

0

There are 0 best solutions below