When I run my testsuite with parallel executions I receive different results compared to run only one test runner.
I tried two approaches and both did not provide proper results. These commands did either only produce one partial result or less than we had runners.
I've tried with coverage and pytest combined:
COVERAGE_PROCESS_START=./my_app coverage --parallel-mode --concurrency=multiprocessing run -m pytest -m "not e2e" -n 4
Also with pytest and pytest-cov:
pytest -m "not e2e" -n 4 --cov=my_app
The second one also had the issue that some templatetags were not seen as registered even though others in the same directory were registered.
After running these I executed coverage combine
and coverage report
. When run in parallel the results are always incomplete compared to running it with only one test runner, which works perfectly fine:
coverage run -m pytest -m "not e2e"
This is my coveragerc:
[run]
include = my_app/*
omit = *migrations*, *tests*
plugins =
django_coverage_plugin
[report]
show_missing = true
Does some know how to get proper coverage results when running pytest in parallel?