Using qExec to create Qt Test suite

Question

QTest encourages you to organize unit tests as separate executables. There is special macro for this, that generates the main function: QTEST_MAIN.

I found this approach not very clean, it is much more useful to run all tests at once. So i searched if there is any possibility of doing so and I found a few people proposing the same solution:

Qt: run unit tests from multiple test classes and summarize the output from all of them

http://www.davideling.it/2014/01/qtest-multiple-unit-test-classes/

https://alexhuszagh.github.io/2016/using-qttest-effectively/

The solution was to give up using QTEST_MAIN macro and write your own main function where you execute the tests you want to execute:

int main(int argc, char *argv[])
{
   int status = 0;

   {
       TestA ta;
       status |= QTest::qExec(&ta, argc, argv);
   }

   {
       TestB tb;
       status |= QTest::qExec(&tb, argc, argv);
   }

   return status;
}

I found it to be a great idea, however, there is a problem. Qt's documentation for qExec has a part that sounds like this:

For stand-alone test applications, this function should not be called more than once, as command-line options for logging test output to files and executing individual test functions will not behave correctly.

The solution revealed by those people suggest just that: executing qExec more than once. Can anyone explain to me what exactly command-line options for logging test output to files and executing individual test functions will not behave correctly exactly mean?

What exactly could go wrong with this approach?

Answer 1

The documentation is probably talking about Logging Options. If you call qMain twice and pass the -o option to both calls, the second call will probably overwrite the log file from the first call. If you know that this will never happen you might choose to ignore the warning. You can also not pass the command line arguments to qExec, that way you will force the output to stdout, but you lose the ability to pass other arguments, of course.

If you want to run the test cases from Qt Creator, you should also not call qExec more than once. Each test class will show up in the test list, but running one will just run all, so you get the result for every class displayed for one class. And if you run all tests (the default), you get a the squared amount of results.

So if you don't like the multiple executable approach, just use Google Test. It has none of the above problems and the Creator provides support for it. The setup is very easy: A wizard will guide you when you create an Autotest project. The only thing you have to do is download Google Test. Google test cases will show up right next to Qt Test cases in the test views.