Detecting specific function calls in unit tests

Question

I would like to be able to detect if my function (or any other function it calls) will end up calling some specific functions (for instance, malloc and free) in my unit tests: some small portions of my software have hard-real-time requirements, and I'd like to ensure that no ones adds something that would trigger an allocation by accident in these functions (and have my CI pipeline check it automatically).

I know that I can just put a breakpoint on gdb, but ideally I'd like to do something like :

void my_unit_test() {
    my_object obj; // perform some initialization that will allocate

    START_CHECKING_FUNCTION(malloc); // also, operator new or std::allocate would be nice
    obj.perform_realtime_stuff();
    STOP_CHECKING_FUNCTION(malloc);
}

ideally, the test would fail in a not-too-dirty way (eg not std::abort) if at some point malloc is called between the two checks.

Ideally, this would run on any system, but I can live with something that only does it on linux for now. Is this possible in some way ? Maybe through a LD_PRELOAD hack that would replace malloc, but I'd rather not have to do this for all the functions I'm interested in.

Answer 1

If you're using libraries which invoke malloc, then you might want to take a look at the Joint Strike Fighter C++ Coding Standards. It's a coding style aimed towards mission critical software. One suggestion would be to write your own allocator(s). Another suggestion is to use something like jemalloc which has statistics, but is much more unpredictable since it is geared towards performance.

---

What you want is a mocking library with spy capabilities. How this works for each framework is going to vary, but here is an example using Google:

static std::function<void*(size_t)> malloc_bridge;

struct malloc_mock {
    malloc_mock() { malloc_bridge = std::bind(&malloc_mock::mock_, this, _1); }
    MOCK_METHOD1(mock_, void*(size_t));
}

void* malloc_cheat(size_t size) {
    return malloc_bridge(size);
}

#define malloc malloc_cheat

struct fixture {
    void f() { malloc(...); }
};

struct CustomTest : ::testing::test {
    malloc_mock mock_;
};

TEST_F(CustomTest, ShouldMallocXBytes) {
    EXPECT_CALL(mock_, mock_(X))
      .WillOnce(::testing::Return(static_cast<void*>(0)));
    Fixture fix;
    fix.f();
}

#undef malloc

WARNING: Code hasn't been touched by compiler hands. But you get the idea.

Answer 2

Unit tests call functions that they test. You want to know if a function F called by a unit test can eventually invoke malloc (or new or ...). Seems like what you really want to do is build a call graph for your entire system, and then ask for the critical functions F whether F can reach malloc etc. in the call graph. This is rather easy to compute once you have the call graph.

Getting the call graph is not so easy. Discovering that module A calls module B directly is "technically easy" if you have a real language front end that does name resolution. Finding out what A calls indirectly isn't easy; you need a (function pointer) points-to analysis and those are hard. And, of course, you have decide if you are going to dive into library (e.g., std::) routines or not.

Your call graph needs needs to be conservative (so you don't miss potential calls) and reasonably precise (so you don't drown in false positives) in the face of function pointers and method calls.

This Doxygen support claims to build call graphs: http://clang.llvm.org/doxygen/CallGraph_8cpp.html I don't know if it handles indirect/methods calls or how precise it is; I'm not very familiar with it and the documentation seems thin. Doxygen in the past did not have reputation for handling indirection well or being precise, but past versions weren't based on Clang. There is some further discussion of this applied on small scale at http://stackoverflow.com/questions/5373714/generate-calling-graph-for-c-code

Your question is tagged c/c++ but seems to be about C++. For C, our DMS Software Reengineering Toolkit with its generic flow analysis and call graph generation support, coupled with DMS's C Front End, has been used to analyze C systems of some 16 million lines/50,000 functions with indirect calls to produce conservatively correct call graphs.

We have not specifically tried to build C++ call graphs for large systems, but the same DMS generic flow analysis and call graph generation would be "technically straightforward" used with DMS's C++ Front End. When building a static analysis that operates correctly and at scale, nothing is trivial to do.

Answer 3

This is not a full answer but you can try to use Valgrind to count allocs and frees. The default Valgrind tool memcheck by default counts the number of allocs and frees and prints resulting report in HEAP SUMMARY, here is a sample output:

$ valgrind ./a.out
==2653== Memcheck, a memory error detector
==2653== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==2653== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==2653== Command: ./a.out
==2653== 
==2653== 
==2653== HEAP SUMMARY:
==2653==     in use at exit: 0 bytes in 0 blocks
==2653==   total heap usage: 2 allocs, 2 frees, 72,716 bytes allocated
==2653== 
==2653== All heap blocks were freed -- no leaks are possible
==2653== 
==2653== For counts of detected and suppressed errors, rerun with: -v
==2653== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

You can add one more test doing nothing to count baseline allocs number:

void my_unit_test_baseline() {
    my_object obj; // perform some initialization that will allocate
}

Now you can run real test and compare the number of allocations with baseline test. If they are not equal than some allocations took place in your tested code. You can log this fact or signal it some other whatever you want way.

Answer 4

In case you are using the GNU C library, you can use the _malloc_hook () and alike functions to have a user-defined function called whenever one of the functions of the malloc family is used.

Such a hooked function could analyse the call trace (using backtrace()) in order to find whether malloc was allowed in this call chain or not and print messages about the culprit if not.