Use a custom failure message for `assertRaises()` in Python?

Question

The Python 2.7 unittest docs say:

All the assert methods (except assertRaises(), assertRaisesRegexp()) accept a msg argument that, if specified, is used as the error message on failure

… but what if I want to specify the error message for assertRaises() or assertRaisesRegexp()?

Use case: when testing various values in a loop, if one fails I’d like to know which one:

NON_INTEGERS = [0.21, 1.5, 23.462, math.pi]

class FactorizerTestCase(unittest.TestCase):
    def test_exception_raised_for_non_integers(self):
        for value in NON_INTEGERS:
            with self.assertRaises(ValueError):
                factorize(value)

If any of these fails, I get:

AssertionError: ValueError not raised

which isn’t too helpful for me to work out which one failed… if only I could supply a msg= argument like I can with assertEqual() etc!

(I could of course break these out into separate test functions — but maybe there are loads of values I want to test, or it requires some slow/expensive setup, or it’s part of a longer functional test)

I’d love it if I could easily get it to report something like:

AssertionError: ValueError not raised for input 23.462

— but it’s also not a critical enough thing to warrant reimplementing/extending assertRaises() and adding a load more code to my tests.

Answer 1

1. Easiest (but hacky!) way to do this I’ve found is:

for value in NON_INTEGERS:
    with self.assertRaises(ValueError) as cm:
        cm.expected.__name__ = 'ValueError for {}'.format(value)  # custom failure msg
        factorize(value)

which will report this on failure:

AssertionError: ValueError for 23.462 not raised

Note this only works when using the with … syntax.

It works because the assertRaises() context manager does this internally:

exc_name = self.expected.__name__
…
raise self.failureException(
    "{0} not raised".format(exc_name))

so could be flaky if the implementation changes, although the Py3 source is similar enough that it should work there too (but can’t say I’ve tried it).

2. Simplest way without relying on implementation is to catch the error and re-raise it with an improved message:

for value in NON_INTEGERS:
    try:
        with self.assertRaises(ValueError) as cm:
            factorize(value)
    except AssertionError as e:
        raise self.failureException('{} for {}'.format(e.message, value)), sys.exc_info()[2]

The sys.exc_info()[2] bit is to reuse the original stacktrace, but this syntax is Py2 only. This answer explains how to do this for Py3 (and inspired this solution).

But this is already making the test hard to read, so I prefer the first option.

The ‘proper’ solution would require writing a wrapped version of both assertRaises AND the _AssertRaisesContext class, which sounds like overkill when you could just throw in some logging when you get a failure.

Answer 2

You could also fallback to using self.fail which feels annoying, but looks a bit less hacky I think

for value in NON_INTEGERS:
    with self.assertRaises(ValueError) as cm:
        factorize(value)
        self.fail('ValueError not raised for {}'.format(value))

Answer 3

I use this instead of assertRaises:

    def test_empty_username(self):
    # noinspection PyBroadException
    try:
        my_func(username="")
    except Exception:
        # If it does, we are OK.
        return
    # If not, we are here.
    self.fail("my_func() must reject empty username.")

Answer 4

In Python 3, unittest now exposes the error message as a proper, publicly accessible property of the _AssertRaisesContext instance you get from with self.assertRaises(). So in Python 3, you can do this:

with self.assertRaises(ValueError) as assertion:
    assertion.msg = f"ValueError not raised for input {value}"
    factorize(value)