parameterization of perl *.t Prove existing tests

Question

I have a bunch of perl tests:

1. Functional Tests
2. Mechanize Tests
3. Actual Unit Tests, asserting on functions and return values, etc
4. Tests that involve external Services, like WebServices
5. DB centric tests
6. REST tests

I run all of them via prove and theoretically re-arrange them into various directories and run something like find t/ -name '*.t' ! -name '*timeout*' | xargs prove -l, but gets very difficult (and not good engineering) to name tests a particular way, so we can parse them via find.

Is there a way we can pass a wildcard list of tests to prove when we run it via command line?
If not, is there a more sane approach than what we're currently using?

Answer 1

The usual way to to this is via environment variables. The test file checks whether it's supposed to run, and if it's not does a quick skip_all.

For example:

use strict;
use warnings;
use Test::More;

BEGIN {
   plan skip_all => "not running extended tests" unless $ENV{EXTENDED_TESTING};
};

# your slow tests here

done_testing();

Now usually that test will be skipped. But if you set the EXTENDED_TESTING environment variable to "1", it will run.

Standard environment variables include EXTENDED_TESTING, RELEASE_TESTING, and NONINTERACTIVE_TESTING. NO_NETWORK_TESTING is also catching on.

There are various modules to automate this such as Test::Is which allows the simpler syntax:

use strict;
use warnings;
use Test::More;
use Test::Is "extended";

# your slow tests here

done_testing();

If you have some other application-specific categories, you'll have to invent some environment variables yourself. If you think they seem generically useful, blog about them, and maybe they'll catch on and become standard environment variables too.

Answer 2

i think i found the answer as Test::Less

Test::Less - Test Categorization and Subset Execution

test-less normally keeps the index file of mappings between tags and test files, in a file called t/Test-Less/index.txt. You can override this with the --file option or the TEST_LESS_INDEX environment variable.

Tags are strings matching /^[\w\-]+$/.

The -list and -prove commands take what is called a tag specification.

A specication is a a list of tags and possibly file names.

test-less -prove foo bar baz

Runs all the foo tests, bar tests and baz tests.

    test-less -prove foo,bar,baz

Even after i was able to fix the compile bug in Test::Less, i was still unable to run any tests using Test::Less which has been broken since 2009. So looking at Test::Class might be the answer:

http://search.cpan.org/~ether/Test-Class-0.46/lib/Test/Class.pm

Sometimes you just want to run a single test. Commenting out other tests or writing code to skip them can be a hassle, so you can specify the TEST_METHOD environment variable. The value is expected to be a valid regular expression and, if present, only runs test methods whose names match the regular expression. Startup, setup, teardown and shutdown tests will still be run.

One easy way of doing this is by specifying the environment variable before the runtests method is called.

Running a test named customer_profile:



#! /usr/bin/perl
 use Example::Test;

 $ENV{TEST_METHOD} = 'customer_profile';
 Test::Class->runtests;

    Running all tests with customer in their name:

     #! /usr/bin/perl
     use Example::Test;

     $ENV{TEST_METHOD} = '.*customer.*';
     Test::Class->runtests;