Embedding Python in C with Generates Import Errors for Python Shared Libraries

Question

The Problem

I've been learning the ins and outs of both Cython and the C API for Python and ran into a peculiar problem. I've created a C program that simply embeds a Python interpreter into it, imports a local .py file, and extracts a function/method from this file. I've been successful in calling a method from a localized file and returning the value to C, so I've gotten the process down to some degree.

I've been working on two different variants: 1) creating a standalone executable and 2) creating a shared library and allowing another executable to access the functions I've built in Python. For case 1, I have no issues. For case 2, I can only successfully make this work if there are no Python dependencies. If my Python function imports a Python core library or any library that is dynamically loaded (e.g. import math, import numpy, import ctypes, import matplotlib) an ImportError will be generated. The paths to these imports are typically in ../pathtopython/lib/pythonX.Y/lib-dynload for the python core libraries/modules.

Some Notes:

I'm using an Anaconda distribution for Python 3.7 on CentOS 7. I do not have administrative access, and there are system pythons that exist that might be interfering - I've demonstrated that I can make this work by using the system python but then I lose the ability to add packages. So I will highlight my findings from using the system python too.

My C Code:

struct results cfuncEval(double height, double diameter){
  
    
    
    Py_Initialize();
    
    PyObject *sys_path, *path;

    sys_path = PySys_GetObject("path");
    
    path = PyUnicode_DecodeFSDefault("path/to/my/python/file/");
    PyList_Insert(sys_path, 0, path);
    
    PyObject *pName, *pModule;
    pName = PyUnicode_DecodeFSDefault("myModule");
    pModule = PyImport_Import(pName);
    
    PyErr_Print();
    PyObject *pFunc = PyObject_GetAttrString(pModule, "cone_funcEval");
    
    
    PyObject* args;
    args = PyTuple_Pack(2,PyFloat_FromDouble(height),PyFloat_FromDouble(diameter));
    
    PyObject* ret        = PyObject_CallObject(pFunc, args);

I do things successfully with ret if there are no import statements in my .py file, however if I import a shared library as mentioned before (such as import math) I will get an error. It's fair to note that I can in fact successfully import other .py files from that .py file without error.

My myModule.py Code:

try:
    import math
except ImportError as e:
    print(e)

def cone_funcEval(diameter, height):
    
    radius = diameter/2.0
  
    volume      = math.pi*radius*radius*height/3.0
    slantHeight = ( radius**2.0 + height**2.0 )**0.5
    surfaceArea = math.pi*radius*(radius + slantHeight)
  
    
    return volume, slantHeight, surfaceArea

Please note that I know this logic is faster in C, but I am just trying to demonstrate creating the linkage and find importing core modules to be an expected feature. So, this is just my example problem. In this .py file, I trigger C to print my ImportError as follows:

/path/to/anaconda3/lib/python3.7/lib-dynload/math.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PyArg_Parse

Indeed Python is not linked to the math .so if I run ldd on it, but I suppose that's to happen at runtime. I can make this work if I create a standalone executable using the -export-dynamic flag with gcc, but I cannot make it work otherwise as that flag appears to be not in use if you are compiling a shared library.

A Word About the System Python:

The system Python's core modules are located in /usr/lib64/pythonX.Y/lib-dynload. When I run ldd on /usr/lib64/pythonX.Y/lib-dynload/math.cpython-XYm-x86_64-linux-gnu.so it identifies the following:

 linux-vdso.so.1 =>  (0x00007fffaf3ab000)
        libm.so.6 => /usr/lib64/libm.so.6 (0x00007f2cd3aff000)
        libpythonX.Ym.so.1.0 => /usr/lib64/libpythonX.Ym.so.1.0 (0x00007f2cd35da000)
        libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007f2cd33be000)
        libc.so.6 => /usr/lib64/libc.so.6 (0x00007f2cd2ff0000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2cd400d000)
        libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007f2cd2dec000)
        libutil.so.1 => /usr/lib64/libutil.so.1 (0x00007f2cd2be9000)

whereas when I run ldd on my anaconda3's math it produces the following:

        linux-vdso.so.1 =>  (0x00007ffe1338a000)
        libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007f1774dc3000)
        libc.so.6 => /usr/lib64/libc.so.6 (0x00007f17749f5000)
        libm.so.6 => /usr/lib64/libm.so.6 (0x00007f17746f3000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1774fdf000)

This, to me, indicates they're compiled somewhat differently. Most notably, the system Python is linked to its libpython.so, whereas my anaconda3 copy is not. I did attempt to compile a brand new version of python from source code using --enable-shared to replicate this to no avail.

My Compilation Flags:

CFLAGS as produced by python3-config --cflags

-I/path/to/anaconda3/include/python3.7m -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem /path/to/anaconda3/include -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -flto -DNDEBUG -fwrapv -O3 -Wall

LDFLAGS

-L/path/to/anaconda3/lib/python3.7/config-3.7m-x86_64-linux-gnu -L/path/to/anaconda3/lib -lm -lrt -lutil -ldl -lpthread -lc -lpython3 -fno-lto

Ideal Solution:

I've searched all over SO and found no solutions that have worked. Ideally I would like to be able to take one of the following solutions:

1. Figure out what flags or steps I need to be able to make my shared library successfully pass the symbols coming from Python.h to the Python symbols needed by the libraries at runtime
2. Figure out how to properly compile a Python from source that looks like the system Python's (or an explanation as to why I can't do that)
3. A good lecturing as to how I have no clue what I'm doing