How to control pytest-cov (or generally any pytest plugin) from my own plugin

Question

I have a plugin myplugin with the following behavior: When calling pytest ... --myplugin=X it should trigger the same behavior as pytest ... --cov=X --cov-report=json.

I'm still new to pytest and while my implementation technically works, I feel very uncomfortable with it because my implementations seems to break pytest behavior (see below) and I cannot manage to find general enough information on the pytest plugin concept in the pytest API reference or tutorials/videos to understand my mistake.

As I'm eager to learn, my question here is twofold

• Concrete: What am I doing wrong in terms of pytest plugin design?
• General: Are there better approaches for controlling another plugin? If yes, how would one apply them to pytest_cov?

We start with an example test project

# myproject/src/__init__.py

def func():
    return 42

# myproject/test_src.py

import src

def test_src():
    assert src.func() == 42

Then there is the plugin

import pytest

def pytest_addoption(parser):
    group = parser.getgroup('myplugin')
    group.addoption(
        '--myplugin',
        action='store',
        dest='myplugin_source',
        default=None,
    )

def _reconfigure_cov_parameters(options):
    options.cov_source = [options.myplugin_source]
    options.cov_report = {
        'json': None
    }

# FIXME this solution to control pytest_cov strongly relies on their implementation details
# - because pytest_cov uses the same hook without hookwrapper,
#   we are guaranteed to come first
# - we modify the config parameters, hence strongly rely on their interface
@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_load_initial_conftests(early_config, parser, args):
    print('\ncode point 1')
    print('early_config.known_args_namespace.cov_source', early_config.known_args_namespace.cov_source)
    print('early_config.known_args_namespace.cov_report', early_config.known_args_namespace.cov_report)
    print('early_config.known_args_namespace.myplugin_source', early_config.known_args_namespace.myplugin_source)

    if early_config.known_args_namespace.myplugin_source is not None:
        _reconfigure_cov_parameters(early_config.known_args_namespace)

    print('\ncode point 2')
    print('early_config.known_args_namespace.cov_source', early_config.known_args_namespace.cov_source)
    print('early_config.known_args_namespace.cov_report', early_config.known_args_namespace.cov_report)
    print('early_config.known_args_namespace.myplugin_source', early_config.known_args_namespace.myplugin_source)

    yield

def pytest_sessionfinish(session, exitstatus):
    print('\ncode point 3')
    print('session.config.option.cov_source=', session.config.option.cov_source)
    print('session.config.option.cov_report', session.config.option.cov_report)
    print('session.config.option.myplugin_source=', session.config.option.myplugin_source)

When I run the plugin, it technically does what it should, cov behaves exactly like I want it producing the json output.

However, if I look at the debug output, it is as follows (truncated to relevant details):

• scenario 1 without myplugin

$ python -m pytest -vs test_src.py --cov=./src --cov-report=html

code point 1
early_config.known_args_namespace.cov_source ['./src']
early_config.known_args_namespace.cov_report {'html': None}
early_config.known_args_namespace.myplugin_source None

code point 2
early_config.known_args_namespace.cov_source ['./src']
early_config.known_args_namespace.cov_report {'html': None}
early_config.known_args_namespace.myplugin_source None

plugins: cov-4.1.0, myplugin-0.1.0

code point 3
session.config.option.cov_source= ['./src']
session.config.option.cov_report {'html': None}
session.config.option.myplugin_source= None

• scenario 2 with myplugin

$ python -m pytest -vs --myplugin=./src

code point 1
early_config.known_args_namespace.cov_source []
early_config.known_args_namespace.cov_report {}
early_config.known_args_namespace.myplugin_source ./src

code point 2
early_config.known_args_namespace.cov_source ['./src']
early_config.known_args_namespace.cov_report {'json': None}
early_config.known_args_namespace.myplugin_source ./src

plugins: cov-4.1.0, myplugin-0.1.0

code point 3
session.config.option.cov_source= []
session.config.option.cov_report {}
session.config.option.myplugin_source= ./src

Coverage JSON written to file coverage.json

So what puzzles me here, is that myplugin seems to break pytests processing of the cov_source option so that my manipulations of cov_ options in early_config.known_args_namespace are not correctly transferred to session.config. Even more surprising is that cov still sees my changes.

That is due to the fact, that cov seems to mainly rely on early_config.known_args_namespace, maybe that is a non-standard paradigm which I shouldn't have followed.

Details:

• plugin structure taken from https://github.com/pytest-dev/cookiecutter-pytest-plugin and installed via pip install -e.
• Python 3.10.12
• pytest-7.3.1
• cov-4.1.0

Answer 1

After looking at the pytest-cov repo it seems your implementation is just about the only way (that I can tell) to modify the parameters before the plugin is configured. Why the debug print outs are different I'm unsure of.

However, I'm going to suggest an alternate approach that may or may not work for your use case, but does solve the debug issue.

Rather than modify the current session, the below code will completely restart the session instead. This also has the potential benefit of relying on the external api (rather than the internal one) which is probably less likely to change.

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_load_initial_conftests(early_config, parser, args):
    session_restart = False
    
    for idx, value in enumerate(args):
        if "myplugin" in value:
            args[idx] = value.replace("myplugin", "cov")
            args.append("--cov-report=json")
            session_restart = True
    
    if session_restart:
        return_code = pytest.main(args)
        exit(return_code)
        
    yield