Using sub-classes with py.test's parametrization

Question

I have a scenario where being able to sub-class a base test class in py.test would make our test environment very extensible. The problem I have is I cannot override the base class's attributes and use them in the parametrize decorator.

import pytest

class TestBase():
    l = [2,3]

    @pytest.mark.parametrize('p', l)
    def testOne(self, p):
        assert p == p

class TestSubClass(TestBase):
    l = [1, 2, 3, 4]

class TestSubClass2(TestBase):
    l = [3, 5]

In this scenario TestSubClass and TestSubClass2 always run using the list l from TestBase because the scope for the decorator to find l is the immediate local scope.

I can't use self.l because self does not exist at the time the decorator is evaluated (there is no instance).

I can work around this and perform the parametrization manually in the test case but then I lose the individual reporting from py.test. Eg.

import pytest
class TestBase()
    def testOne(self):
        for p in self.l:
            assert p == p

class TestSubClass(TestBase):
    l = [1, 2, 3, 4]

class TestSubClass2(TestBase):
    l = [3, 5]

How can I sub-class a base class and customize the parametrisation for each sub-class?

Answer 1

I have a slightly terrible solution:

1. Change the base class to have the list as a super-set of all possible options.
2. At collection time py.test builds the list of all possible tests
3. At run time check to see if the value for the parameter is a valid one for that base class. If it is not, skip the test

The code:

import pytest

class TestBase():
    l = [1, 2, 3, 4, 5]

    @pytest.mark.parametrize('p', l)
    def testOne(self, p):
        if p not in self.l:
            pytest.skip("%d not supported on %s" % (p, self.__class__.__name__))
        assert p == p

class TestSubClass(TestBase):
    l = [1, 2]

class TestSubClass2(TestBase):
    l = [3, 5]

And the corresponding output:

py.test -v -rsf test_sample.py::TestSubClass
===================================== test session starts ======================================
platform darwin -- Python 2.7.9 -- py-1.4.26 -- pytest-2.6.4 -- 
collected 5 items 

test_sample.py::TestSubClass::testOne[1] PASSED
test_sample.py::TestSubClass::testOne[2] PASSED
test_sample.py::TestSubClass::testOne[3] SKIPPED
test_sample.py::TestSubClass::testOne[4] SKIPPED
test_sample.py::TestSubClass::testOne[5] SKIPPED
=================================== short test summary info ====================================
SKIP [1] test_sample.py:10: 4 not supported on TestSubClass
SKIP [1] test_sample.py:10: 3 not supported on TestSubClass
SKIP [1] test_sample.py:10: 5 not supported on TestSubClass

============================= 2 passed, 3 skipped in 0.01 seconds ==============================

There's some terrible consequences of this approach:

• Adding a single specific case for a new sub-class requires that you add it to both the sub-class and the base class
• If you don't add the specific case to the base class it will silently not run it.
• The test case needs a whole lot of extra 'look before you leap' code at the start to determine if it is a valid test case