Delphi 7 Dunit checks after StopExpectingException are not working as I expect

Question

The below code works fine, the calc... generates an exception, comment it out or change calc... to not throw and exception and the test fails.

  StartExpectingException(exception);
  calcMembersPIPEndDate(EncodeDate(2005,01,01),true);
  StopExpectingException('calcMembersPIPEndDate - 1st after aDay');

My problem is that any checks I put in this test method after this do not execute.
so

  checkEquals(1,0);
  StartExpectingException(exception);
  calcMembersPIPEndDate(EncodeDate(2005,01,01),true);
  StopExpectingException('calcMembersPIPEndDate - 1st after aDay');

fails on the 1st checkEquals

  StartExpectingException(exception);
  calcMembersPIPEndDate(EncodeDate(2005,01,01),true);
  StopExpectingException('calcMembersPIPEndDate - 1st after aDay');
  checkEquals(1,0);

passes - why?

I have tried to work out what version of Dunit I am using:

testframework.pas has the following - which didn't seem to 
rcs_id: string = '#(@)$Id: TestFramework.pas,v 1.117 2006/07/19 02:45:55
rcs_version : string = '$Revision: 1.117 $';
versioninfo.inc
ReleaseNo : array[1..3] of Integer
          = (9,2,1);
ReleaseStr     = '9.2.1';
ReleaseWhen : array[1..6] of Integer
          = (2005,09,25,17,30,00);

Answer 1

StopExpectingException cannot work the way you expect. It's important to understand the flow of execution in an exception state to see why.

Consider the following code:

procedure InnerStep(ARaiseException);
begin
  Writeln('Begin');
  if ARaiseException then
    raise Exception.Create('Watch what happens now');
  Writeln('End');
end;

procedure OuterStep;
begin
  try
    InnerStep(False); //1
    InnerStep(True);  //2
    InnerStep(False); //3
  except
    //Do something because of exception
    raise;
  end;
end;

When you call OuterStep above, line //2 will raise an exception inside InnerStep. Now whenever an exception is raised:

• The instruction pointer jumps out of each method (a little like goto) to the first except or finally block found in the call-stack.
• Writeln('End'); will not be called.
• Line //3 will not be called.
• Whatever code exists in the except block of OuterStep is executed next.
• And finally when raise; is called, the exception is re-raised and the instruction pointer jumps to the next except or finally block.
• Note also that like raise; any other exception within the except block will also jump out, (effectively hiding the first exception).

So when you write:

StartExpectingException(...);
DoSomething();
StopExpectingException(...);

There are 2 possibilities:

1. DoSomething raises an exception and StopExpectingException is never called.
2. DoSomething doesn't raise an exception and when StopExpectingException is called there is no exception.

---

David has explained that the DUnit framework calls StopExpectingException for you. But you may be wondering how to approach your test case checking multiple exception scenarios.

Option 1

Write smaller tests.
You know that's what everyone says you're supposed to do in any case right? :)
E.g.

procedure MyTests.TestBadCase1;
begin
  ExpectedException := ESomethingBadHappened;
  DoSomething('Bad1');
  //Nothing to do. Exception should be raised, so any more code would
  //be pointless.
  //If exception is NOT raised, test will exit 'normally', and
  //framework will fail the test when it detects that the expected
  //exception was not raised.
end;

procedure MyTests.TestBadCase2;
begin
  ExpectedException := ESomethingBadHappened;
  DoSomething('Bad2');
end;

procedure MyTests.TestGoodCase;
begin
  DoSomething('Good');
  //Good case does not (or should not) raise an exception.
  //So now you can check results or expected state change.
end;

Option 2

As David has suggested, you can write your own exception handling inside your test. But you'll note that it can get a little messy, and you'll probably prefer option 1 in most cases. Especially when you have the added benefit that distinctly named tests make it easier to identify exactly what went wrong.

procedure MyTests.TestMultipleBadCasesInTheSameTest;
begin
  try
    DoSomething('Bad1');
    //This time, although you're expecting an exception and lines
    //here shouldn't be executed:
    //**You've taken on the responsibility** of checking that an
    //exception is raised. So **if** the next line is called, the
    //expected exception **DID NOT HAPPEN**!
    Fail('Expected exception for case 1 not raised');
  except
    //Swallow the expected exception only!
    on ESomethingBadHappened do;
    //One of the few times doing nothing and simply swallowing an
    //exception is the right thing to do.
    //NOTE: Any other exception will escape the test and be reported
    //as an error by DUnit
  end;

  try    
    DoSomething('Bad2');
    Fail('Expected exception for case 2 not raised');
  except
    on E: ESomethingBadHappened do
      CheckEquals('ExpectedErrorMessage', E.Message);
      //One advantage of the manual checking is that you can check
      //specific attributes of the exception object.
      //You could also check objects used in the DoSomething method
      //e.g. to ensure state is rolled back correctly as a result of
      //the error.
  end;
end;

NB! NB! Something very important to note in option 2. You need to be careful about what exception class you swallow. DUnit's Fail() method raises an ETestFailure exception to report to the framework that the test failed. And you wouldn't want to accidentally swallow the exception that's going to trigger the test failure for expected exception.

The subtle issues related exception testing make it important to: test first, ensure you have the correct failure, and only then implement the production code change to get a pass. The process will significantly reduce the chances of a dud test.

Answer 2

These two methods, StartExpectingException and StopExpectingException are not meant to be called directly.

Instead you are supposed to use the ExpectedException property. When you set this property, StartExpectingException is called. Whilst you could call StartExpectingException I belive that the intended usage is that you assign to ExpectedException.

As for StopExpectingException, you don't call it. The framework calls it. It does so in TTestCase.RunTest, the framework code that executes your test method.

So your test case code might look like this:

ExpectedException := ESomeException;
raise ESomeException.Create(...);

When you state that you are expecting an exception, what you are saying is that your test method will raise that exception. Since raising an exception alters control flow, code that appears after the exception is raised will not execute. Exceptions propagate up the call stack until they are caught. The framework will catch the exception in TTestCase.RunTest. If you have indicated that the caught exception is expected then the test will pass, otherwise failure is recorded.

The net result of all this is that the ExpectedException mechanism can be used if the final act of the test method is to raise that expected exception. The ExpectedException mechanism is no use at all if you want to perform further tests after the exception is raised. If you wish to do that then you should either:

1. Write your own exception handling code, in your test method, that checks that exceptions are raised as designed.
2. Use CheckException.