How do I test C functions with internal ifdefs for functional equivalency?

67 Views Asked by At

I have a library of C functions that I optimized internally using SIMD intrinsics. These functions all look something like this:

void add_array(...) {
#if defined(USE_SIMD)
  // SIMD code here ...
#else 
  // Scalar code here ...
#end
}

contained in individual files, so a add.c file here, for example.

Now, I would like to ensure that both variants of that function are functionally equivalent. I found that simply generating random (but valid) input for both variants and comparing the results suffices for my application. I think this is called Monkey Testing. The scalar code (or rather its output values) acts as a golden reference. (Because of the SIMD intrinsics formal verification is not an option.)

However, I have not found a scalable and sustainable way to run these tests from a C testing framework. So far, I manually copied the vector code into an extra vector function add_array_vector() and then ran them both after another from the same main function test harness; comparing the "golden reference output" from the add_array() function with the values from the add_array_vector() variant. But this approach does not scale, since I have more than 100 of these functions that all use the #if #else approach internally. Since I have to run all code in a simulator (or on a bare-metal embedded device), I also can't interact with a file system. I need a single test binary that contains all tests and test data. It has to report its results via a printf (UART) call.

What I see as my only option is to compile the functions twice: Once without the USE_SIMD and once with the USE_SIMD defined. Then I would need to link these two function variants into the same main (my test harness). However, how do I ensure that both variants have different function names? Is there a way I can "name mangle" the USE_SIMD define into the function name? And how would I link them?

Maybe I am completely on the wrong track here and there is a far simpler way to solve this. I surely can't be the first person who came across this core issue: ensuring that two variants of the same C (or C++) function are functionally equivalent.

Any help is greatly appreciated. Thanks

EDIT: I can't afford to print the numeric results via printfs (or UART) as they are a serious bottleneck in this randomized bruteforce approach. They dramatically reduce (multiple order of magnitude) the number of iterations / tests I can run per second. Printing the final outcome or an error, if one occurs, is fine. Printing every numerical test result value for "external validation" is not sustainable.

0

There are 0 best solutions below