CUnit failures in hudson show success

Question

i put unit tests in C running over hudson, but when i have a test that failures it shows a success in the build, but must to appear fail!

I'm using gcc to compile and run the executable in hudson like that:

gcc -lcunit test_ctesting.c -o testing
./testing

it's in a shell command, how can i get the failures and show it?

Answer 1

The exit code is stored in the variable $?, and I'm assuming that if tests fail it's set to something non-zero.

... if it's something unixy, anyway.

Answer 2

You can use CUnit XML to report your test result. Then use "Publish JUnit test result report" in Jenkins to show the result.

See CUnit automated for XML output: http://cunit.sourceforge.net/doc/running_tests.html#automated

Answer 3

I'm not sure if this is what you're looking for, but the CUnit example code does this:

   CU_basic_set_mode(CU_BRM_VERBOSE);
   CU_basic_run_tests();
   CU_cleanup_registry();
   return CU_get_error();

I can't remember exactly what CU_get_error() returns right now, but it doesn't return a non-zero value if you've had tests fail. If you want to figure that out, you need to return the number of tests failed:

   unsigned int num_failures;
   CU_basic_set_mode(CU_BRM_VERBOSE);
   CU_basic_run_tests();
   num_failures = CU_get_number_of_failures();
   CU_cleanup_registry();
   return num_failures;

I'm not sure how Hudson/Jenkins deals with this, but the non-zero return value will have CMake/CTest not report passed tests.

Answer 4

You can tell CUnit to run automated as Bjerking suggests.

It generates an Xml named CUnitAutomated-Results.xml at default.

This xml has to be transformed to be read from the "Publish JUnit test result report" task in Jenkins.

Steps:

• in your test program (ie test.c) call

  CU_automated_run_tests();

• Make and invoke your test program from Jenkins with a command like this:

  ./test

• Get CUnit-to-JUnit transform and put it on your build machine

• Transform CUnitAutomated-Results.xml to Test-Result.xml using cunit-to-junit.xsl:

  xsltproc --novalid ../Build/cunit-to-junit.xsl CUnitAutomated-Results.xml > Test-Results.xml

• Add a "Publish JUnit test result report" task and point to:

  Test-Results.xml

Now when you build your project you get:

• SUCCESS if all tests are green
• UNSTABLE if you have one failure at least
• full Test Result Trend