Negative Property-Based Tests

Question

Property-based testing is good when you can express simple and well-defined properties.

I've also had luck with "negative properties" in the case of testing parsers, e.g. by generating invalid identifiers or mismatching indentation for indentation-sensitive languages. But is "negative properties" a common thing to express and test for? It means writing generators that produce specialized, bogus input.

What are some real-life examples of negative properties not related to parsers?

Answer 1

I have essentially the same question.

I've posted it in more detail, along with all of my thinking thus far, on how this might be accomplished:

Pattern for generating negative Scalacheck scenarios: Using property based testing to test validation logic in Scala

For us, the "real world" situation is that we have large, data-bound case classes, and those case classes have validation logic (we use Wix Accord for the validation). To test the validation logic thoroughly, we need to create an object, invalidate one of its property, and see if validate(o) fails. Then repeat for each and every property on the object.

We also run into specific cases where a specific property needs to be invalid. For instance, checking to see if our system will correctly handle an invalid ID. That's easier, we can just use a forAll(someGen) and mutate it. Effectively, just v => val invalidV = v copy(id = "badID") and now for every generated property, I have mutated it to have a bad ID. You can get fancier of course, but you get the idea.

All of this is summarized in my other post... along with open solicitation for "the best pattern." Hoping to get some good ideas!