Why does my Perl unit test fail in EPIC but work in the debugger?

Question

Has anyone ever experienced a unit test that fails and when they tried to debug it to find out where the failure was occurring, the unit test succeeds when running the code in the debugger?

I'm using Eclipse 3.5.1 with EPIC 0.6.35 and ActiveState ActivePerl 5.10.0. I wrote module A and module B both with multiple routines. A routine in module B calls a bunch of routines from module A. I'm adding mock objects to my module B unit test file to try to get more complete code coverage on module B where the code in module B tests to see if all the calls to module As routines fail or succeed. So I added some mock objects to my unit test to force some of the module A routines to return failures, but I was not getting the failures as expected. When I debugged my unit test file, the calls to the module A routine did fail as expected (and my unit test succeeds). When I run the unit test file as normal without debugging, the call to the mocked Module A routine does not fail as expected (and my unit test fails).

What could be going on here? I'll try to post a working example of my problem if I can get it to fail using a small set of simple code.

ADDENDUM: I got my code whittled down to a bare minimum set that demonstrates my problem. Details and a working example of the problem follows:

My Eclipse project contains a "lib" directory with two modules ... MainModule.pm and UtilityModule.pm. My Eclipse project also contains at the top level a unit test file named MainModuleTest.t and a text file called input_file.txt which just contains some garbage text.

EclipseProject/
    MainModuleTest.t
    input_file.txt
    lib/
        MainModule.pm
        UtilityModule.pm

Contents of the MainModuleTest.t file:

use Test::More qw(no_plan);
use Test::MockModule;
use MainModule qw( mainModuleRoutine );

$testName = "force the Utility Module call to fail";
# set up mock utility routine that fails
my $mocked = new Test::MockModule('UtilityModule');
$mocked->mock( 'slurpFile', undef );
# call the routine under test
my $return_value = mainModuleRoutine( 'input_file.txt' );
if ( defined($return_value) ) {
    # failure; actually expected undefined return value
    fail($testName);
}
else {
    # this is what we expect to occur
    pass($testName); 
}

Contents of the MainModule.pm file:

package MainModule;

use strict;   
use warnings; 
use Exporter; 
use base qw(Exporter); 
use UtilityModule qw( slurpFile );

our @EXPORT_OK = qw( mainModuleRoutine );

sub mainModuleRoutine {
    my ( $file_name ) = @_;
    my $file_contents = slurpFile($file_name);
    if( !defined($file_contents) ) {
        # failure
        print STDERR "slurpFile() encountered a problem!\n";
        return;
    }
    print "slurpFile() was successful!\n";
    return $file_contents;
}

1;  

Contents of the UtilityModule.pm file:

package UtilityModule;

use strict;   
use warnings; 
use Exporter; 
use base qw(Exporter); 

our @EXPORT_OK = qw( slurpFile );

sub slurpFile {
    my ( $file_name ) = @_;
    my $filehandle;
    my $file_contents = "";
    if ( open( $filehandle, '<', $file_name ) ) {
        local $/=undef;
        $file_contents = <$filehandle>;
        local $/='\n';
        close( $filehandle ); 
    }
    else {
        print STDERR "Unable to open $file_name for read: $!";
        return;    
    } 
    return $file_contents;
}

1;   

When I right-click on MainModuleTest.t in Eclipse and select Run As | Perl Local, it gives me the following output:

slurpFile() was successful!
not ok 1 - force the Utility Module call to fail
1..1
#   Failed test 'force the Utility Module call to fail'
#   at D:/Documents and Settings/[SNIP]/MainModuleTest.t line 13.
# Looks like you failed 1 test of 1.

When I right click on the same unit test file and select Debug As | Perl Local, it gives me the following output:

slurpFile() encountered a problem!
ok 1 - force the Utility Module call to fail
1..1

So, this is obviously a problem. Run As and Debug As should give the same results, right?!?!?

Answer 1

Both Exporter and Test::MockModule work by manipulating the symbol table. Things that do that don't always play nicely together. In this case, Test::MockModule is installing the mocked version of slurpFile into UtilityModule after Exporter has already exported it to MainModule. The alias that MainModule is using still points to the original version.

To fix it, change MainModule to use the fully qualified subroutine name:

 my $file_contents = UtilityModule::slurpFile($file_name);

The reason this works in the debugger is that the debugger also uses symbol table manipulation to install hooks. Those hooks must be getting installed in the right way and at the right time to avoid the mismatch that occurs normally.

It's arguable that it's a bug (in the debugger) any time the code behaves differently there than it does when run outside the debugger, but when you have three modules all mucking with the symbol table it's not surprising that things might behave oddly.

Answer 2

Does your mocking manipulate the symbol table? I've seen a bug in the debugger that interferes with symbol table munging. Although in my case the problem was reversed; the code broke under the debugger but worked when run normally.