Using fixtures at collect time in pytest

Question

I use testinfra with ansible transport. It provides host fixture which has ansible, so I can do host.ansible.get_variables().

Now I need to create a parametrization of test based on value from this inventory.

Inventory:

foo:
  hosts:
    foo1:
      somedata:
        - data1
        - data2

I want to write a test which tests each of 'data' from somedata for each host in inventory. 'Each host' part is handled by testnfra, but I'm struggling with parametrization of the test:

@pytest.fixture
def somedata(host):
    return host.ansible.get_variables()["somedata"]

@pytest.fixture(params=somedata):
def data(request):
    return request.param


def test_data(host, data):
   assert 'data' in data

I've tried both ways:

• @pytest.fixture(params=somedata) -> TypeError: 'function' object is not iterable
• @pytest.fixture(params=somedata()) -> Fixture "somedata" called directly. Fixtures are not meant to be called directly...

How can I do this? I understand that I can't change the number of tests at test time, but I pretty sure I have the same inventory at collection time, so, theoretically, it can be doable...

Answer 1

You should use helper function instead of fixture to parametrize another fixture. Fixtures can not be used as decorator parameters in pytest.

def somedata(host):
    return host.ansible.get_variables()["somedata"]

@pytest.fixture(params=somedata()):
def data(request):
    return request.param


def test_data(host, data):
   assert 'data' in data

This assumes that the host is not a fixture.

If the host is a fixture, there is hacky way to get around the problem. You should write the parameters to a tmp file or in a environment variable and read it with a helper function.

import os

@pytest.fixture(autouse=True)
def somedata(host):
    os.environ["host_param"] = host.ansible.get_variables()["somedata"]

def get_params():
    return os.environ["host_param"] # do some clean up to return a list instead of a string    


@pytest.fixture(params=get_params()):
def data(request):
    return request.param


def test_data(host, data):
   assert 'data' in data

Answer 2

After reading a lot of source code I have came to conclusion, that it's impossible to call fixtures at collection time. There are no fixtures at collection time, and any parametrization should happen before any tests are called. Moreover, it's impossible to change number of tests at test time (so no fixture could change that).

Answering my own question on using Ansible inventory to parametrize a test function: It's possible, but it requires manually reading inventory, hosts, etc. There is a special hook for that: pytest_generate_tests (it's a function, not a fixture).

My current code to get any test parametrized by host_interface fixture is:

def cartesian(hosts, ar): 
    for host in hosts:
        for interface in ar.get_variables(host).get("interfaces",[]):
            yield (host, interface)

def pytest_generate_tests(metafunc):
    if 'host_interface' in metafunc.fixturenames:
        inventory_file = metafunc.config.getoption('ansible_inventory')
        ansible_config = testinfra.utils.ansible_runner.get_ansible_config()
        inventory = testinfra.utils.ansible_runner.get_ansible_inventory(ansible_config, inventory_file)
        ar = testinfra.utils.ansible_runner.AnsibleRunner(inventory_file)
        hosts = ar.get_hosts(metafunc.config.option.hosts)
        metafunc.parametrize("host_interface", cartesian(hosts, ar))