Python Hypothesis - building strategy once for many tests?

948 Views Asked by At

I have a composite, expensive-to-build but cheap-to-test strategy. I must do:

@given(expensive_strategy())
def test_all(x):
    assert...
    assert...
    ...

It takes ~4 seconds to build the examples and negligible time to run the asserts.

Best practice dictates these tests be separated.

It is not clear to me how I could combine hypothesis strategies and eg. TestCase.setUp or pytest session-scoped fixtures. Both decorating the fixture and calling with x = expensive_strategy(); @given(x) does not help.

1

There are 1 best solutions below

2
On

One option would be to use the strategy solely for example generation, and ignore testcase shrinking, the examples database, etc. This will make a passing test suite faster, and a failing test suite more confusing. Something like the following should work:

class MyTest(TestCase):
    @classmethod  # or however this is done in your test runner
    def setUpClass(cls):
        strategy = expensive_strategy()
        cls.examples = [strategy.example() for _ in range(1000)]

    def test_all(self):
        for x in self.examples:
            assert invariant(x)

Another option would be to mangle hypothesis' internals so that each testcase receives the same bytestream, and do caching on whatever is slow in constructing the strategy, though that would surely violate "best practices".

Personally, I would just not follow "best practices" when they lead to absurd results like a slow test suite, and make all the assertions in a single test method, possibly via calls to helper methods like assert_a_particular_kind_of_assertion(x).