jUnit fail() conventions

Question

I am wondering, by convention, when a test fails, is it appropriate to:

• Say why it failed (business logic)
• Say why the message is seen (exception should have been thrown and it's not)

For instance,

fail("Accessed the element which does not exist");

or

fail("ArrayIndexOutOfBoundException was expected but something bad happened");

Which one is generally preferred/ accepted?

Answer 1

Given that this is testing code and shouldn't make it into the final build, I'd say the more verbose the better. As such, I'd go with the first one - it's much clearer what the root cause of the problem is, which makes it easier to correct.

I'd avoid the second one under all circumstances because it's not very helpful for a tester, and if it does somehow make it into the final build its even more cryptic than the first to an end user - it simply is no help for testers or users alike.

The one other option that I'd consider is rather than indicating that an exception occurred, instead give details of the actual exception - that's why stack traces were invented. That would convey more detail than either method listed.

Answer 2

If there were a convention about this I would ignore it. You should say whatever will best communicate to someone seeing this message the nature of the problem in such a way that it can be resolved as easily as possible. Too often adhering to a convention fails in this regard.

Answer 3

First, if you expect the API you test to throw an exception, instead of doing a try-catch with a fail()...

@Test
public void testMe() {
   try {
      api.testThis();
      fail("should not reach here");
   }
   catch(MyException e) {}
}

... you should do this:-

@Test(expected=MyException.class)
public void testMe() {
   api.testThis();
}

That said, I rarely use fail(). If I need to perform certain validation and the condition fails, I will most likely use assertions than using fail()... for example:-

... instead of...

@Test
public void testMe() {
   boolean bool = api.testThis();
   if (!bool) {
      fail("should be true because bla bla bla");
   }
}

... do this:-

@Test
public void testMe() {
   boolean bool = api.testThis();
   assertTrue("bla bla bla",bool);
}

However, if you really need to use fail(), be verbose and explain why it fails rather than "should not reach here" or "should fail here" because that doesn't help folks who read the testcases.

Granted, these are just simple examples.... but I think I get my points across. :)