Querying Python runtime for all objects in existence

Question

I'm working on a C++ Python wrapper the attempts to encapsulate the awkwardness of reference counting, retaining, releasing.

It has a set of unit tests.

However I want to ensure that after each test, everything has been cleared away properly. i.e. every object created during that test has had its reference count taken down to 0, and has consequently been removed.

Is there any way of querying the Python runtime for this information?

If I could just get the number of objects being stored, that would do. I could then sure it doesn't change between tests.

EDIT: I believe it is possible to compile Python with a special flag producing a binary that has functions for monitoring reference counting. But this is as much as I know. Maybe more...

Answer 1

That depends on which implementation you use. I'm assuming you're using cpython. Since you're fiddling with the reference counting mechanism, I will further assume that using the garbage collector to find the remaining objects won't be sufficiently reliable for your purpose. (Elsewise, see here.)

The build flag you were thinking about is this one:

It is best to define these options in the EXTRA_CFLAGS make variable:

make EXTRA_CFLAGS="-DPy_REF_DEBUG".

Py_REF_DEBUG introduced in 1.4 named REF_DEBUG before 1.4

Turn on aggregate reference counting. This arranges that extern _Py_RefTotal hold a count of all references, the sum of ob_refcnt across all objects. [..] Special gimmicks: sys.gettotalrefcount() Return current total of all refcounts.

(Source: Python git, SpecialBuilds.txt, Debugging builds from the C API reference.)

If you need a list of all pointers to live objects, use Py_TRACE_REFS, directly below that one in the SpecialBuilds file.