Cython: calling C function throws 'undefined symbol'

Question

I am attempting to use the LMDB C API with Cython.

I want to import the following definitions from the header file:

typedef struct MDB_env MDB_env;
int  mdb_env_create(MDB_env **env);

So I created a .pxd file:

cdef extern from 'lmdb.h':
    struct MDB_env:
        pass
    int  mdb_env_create(MDB_env **env)

And I am using it in a Cython script:

cdef MDB_env *e
x = mdb_env_create(&e)

This code compiles fine, but If I run it, I get:

ImportError: /home/me/.cache/ipython/cython/_cython_magic_15705c11c6f56670efe6282cbabe4abc.cpython-36m-x86_64-linux-gnu.so: undefined symbol: mdb_env_create

This happens both in a Cython .pyx + .pxd setup and in a prototype typed in IPython.

If I import another symbol, say a constant, I can access it. So I seem to be looking at the right header file.

I don't see any discrepancy between my syntax and the documentation, but I am clearly doing something wrong. Can somebody give me a hint?

Thanks.

Answer 1

To compile it with IPythons-magic (would be nice if you would mention this explicitly in your question) you have to provide library-path (via -L-option) and library name (via -l-option) of the built c-library you want to wrap, see also the documentation:

%%cython  -L=<path to your library> -l=<your_library>

The library you are trying to wrap is not a header-only library. That means that some symbols (e.g. mdb_env_create) are only declared but not defined in the header. When you build the library, the definitions of those symbols can be found in the resulting artifact, which should be provided to the linker when your extension is built. These definitions is what is needed when the program runs.

If you don't do it, the following happens on Linux: When the extension (the *.so-file) is built,the linker allows undefined symbols per default - so this step is "successful" - but the failure is only postponed. When the extension is loaded via import, Python loads the corresponding *.so with help of ldopen and in this step loader checks that the definitions of all symbols are known. But we didn't provide a definition of mdb_env_create so, the loader fails with

undefined symbol: mdb_env_create

It is differently for symbols which are defined in the header-file, for example enums MDB_FIRST&Co - the compiled library isn't necessary and thus the extension can be loaded, as there are no undefined symbols.