Logtalk : what is the best way to run all test suites?

Question

In Logtalk code examples, each example provides its own test suite which is runnable in a "standalone" mode (one test suite at once).

But, as the title says, I'm interested in the best approaches of testing all test suites (all loaded objects inheriting lgtunit in my app) at once, and having one single summary of all tests execution at the end (total passed / skipped / failed).

For example, in SWI-Prolog, run_tests/0 run all test-units.

Answer 1

Here is a first implementation of a runner object to run all registered test suites : https://github.com/koryonik/logtalk-experiments/tree/master/test-runner

Usage is simple :

Simply run all loaded lgtunit test suites :

test_runner::autoregister_tests, % register all loaded lgtunit objects
test_runner::run_tests.

Or Manually register test suites you want to run :

test_runner::register_tests(test_suite_obj1),
test_runner::register_tests(test_suite_obj2),
test_runner::run_tests. %run the 2 test suites

Answer 2

For automation, there's a logtalk_tester Bash shell script included in the Logtalk distribution that gives you a single summary. For running all tests for all loaded objects extending lgtunit, a partial solution could be a goal such as:

?- forall(extends_object(TestObject, lgtunit), TestObject::run).

But this will not give you a single summary. A solution is to define a summary object defining the logtalk::message_hook/4 hook predicate to intercept and collect all relevant information and then to summarize it. You can check the message terms in the lgtunit/lgtunit_messages.lgt file. The one that you want to intercept is tests_results_summary(Total, Skipped, Passed, Failed, Note). Something like:

:- object(test_summary).

    :- public(report/0).
    report :-
        % compute totals from result_/4 and report them
        ...

    :- private(result_/4).
    :- dynamic(result_/4).

    :- multifile(logtalk::message_hook/4).
    :- dynamic(logtalk::message_hook/4).
    logtalk::message_hook(tests_results_summary(Total,Skipped,Passed,Failed,_), _, lgtunit, _) :-
        assertz(result_(Total,Skipped,Passed,Failed)).

:- end_object.

Possibly, also add a run_all/0 predicate to this object using e.g. the solution above. The multifile predicate is also dynamic. Thus, you can assert and retract its definition so that it would only be active when you want to run all tests and summarize the results.

Btw, a fully developed and document solution along the lines above would make a nice contribution to Logtalk...