Code coverage in SimpleTest

Question

Is there any way to generate code coverage report when using SimpleTest similar to PHPUnit.

I have read the documentation of SimpleTest on their website but can not find a clear way on how to do it! I came across this website that says

we can add require_once (dirname(__FILE__).'/coverage.php') to the intended file and it should generate the report, but it did not work!

If there is a helpful website on how to generate code coverage, please share it here.

Thanks alot.

Answer 1

I could not get it to work in the officially supported way either, but here is something I got working that I was able to hack together by examining their code. This works for v1.1.7 of SimpleTest, not their master code. At the time of this writing v1.1.7 is the latest release, and works with new versions of PHP 7, even though it is an old release.

First off you have to make sure you have Xdebug installed, configured, and working. On my system there is both a CLI and Apache version of the php.ini file that have to be configured properly depending on if I am trying to use PHP through Apache or just directly from the terminal. There are alternatives to Xdebug, but most people us Xdebug.

Then, you have to make the PHP_CodeCoverage library accessible from your code. I recommend adding it to your project as a composer package.

Now you just have to manually use that library to capture code coverage and generate a report. How exactly you do that will depend on how you run your tests. Personally, I run my tests on the terminal, and I have a bootstrap file that php runs before it starts the script. At the end of the bootstrap file, I include the SimpleTest autorun file so it will automatically run the tests in any test classes that get included like so:

require_once __DIR__.'/vendor/simpletest/simpletest/autorun.php';

Somewhere inside your bootstrap file you will need to create a filter, whitelist the directories and files you want to get reported, create a coverage object and pass in the filter to the constructor, start coverage, and create and register a shutdown function that will change the way SimpleTest executes the tests to make sure it also stops the coverage and generates the coverage report. Your bootstrap file might look something like this:

<?php
require __DIR__.'/vendor/autoload.php';
$filter = new \SebastianBergmann\CodeCoverage\Filter();
$filter->addDirectoryToWhitelist(__DIR__."/src/");
$coverage = new \SebastianBergmann\CodeCoverage\CodeCoverage(null, $filter);
$coverage->start('<name of test>');

function shutdownWithCoverage($coverage)
{
    $autorun = function_exists('\run_local_tests'); // provided by simpletest
    if ($autorun) {
        $result = \run_local_tests(); // this actually runs the tests
    }
    $coverage->stop();
    $writer = new \SebastianBergmann\CodeCoverage\Report\Html\Facade;
    $writer->process($coverage, __DIR__.'/tmp/code-coverage-report');
    if ($autorun) {
        // prevent tests from running twice:
        exit($result ? 0 : 1);
    }
}
register_shutdown_function('\shutdownWithCoverage', $coverage);
require_once __DIR__.'/vendor/simpletest/simpletest/autorun.php';

Answer 2

It took me some time to figure out, as - to put it mildly - the documentation for this feature is not really complete.

Once you have your test suite up and running, just include these lines before the lines that are actually running it:

require_once ('simpletest/extensions/coverage/coverage.php');
require_once ('simpletest/extensions/coverage/coverage_reporter.php');

$coverage = new CodeCoverage();
$coverage->log = 'coverage/log.sqlite'; // This folder should exist
$coverage->includes = ['.*\.php$'];     // Modify these as you wish
$coverage->excludes = ['simpletest.*']; // Or it is even better to use a setting file
$coverage->maxDirectoryDepth = '1';
$coverage->resetLog();
$coverage->startCoverage();

Then run your tests, for instance:

$test = new ProjectTests(); //It is an extension of the class TestSuite
$test->run(new HtmlReporter());

Finally generate your reports

$coverage->stopCoverage();
$coverage->writeUntouched();

$handler           = new CoverageDataHandler($coverage->log);
$report            = new CoverageReporter();
$report->reportDir = 'coverage/report'; // This folder should exist
$report->title     = 'Code Coverage Report';
$report->coverage  = $handler->read();
$report->untouched = $handler->readUntouchedFiles();
$report->summaryFile = $report->reportDir . '/index.html';

And that's it. Based on your setup, you might need to make some small adjustment to make it work. For instance, if you are using the autorun.php from simpletest, that might be a bit more tricky.