Can't link shared library with -mx32 and gcc 4.7 or gcc 4.8

Question

I'm trying to compile a large codebase written for a 32 bit embedded processor to run on a 64 bit desktop processor for simulation/unit testing. I need the resulting object to be a shared library. This is not an issue in Windows, I can build a dll like this (/DWIN32) and it runs fine.

In Linux, I'm able to compile and link with the -m32 option given to both gcc and the linker and get a shared library out. The problem is, this library is (just like I specified with -m32) a 32 bit library and will not run on my 64 bit arch. With Python, I try to load the library (using ctypes.cdll.LoadLibrary())

OSError: out.so: wrong ELF class: ELFCLASS32

I discovered the -mx32 option, and according to the docs this is exactly what I want:

The -mx32 option sets int, long, and pointer types to 32 bits, and generates code for the x86-64 architecture.

So, I pass -mx32 to the compiler and the linker (replacing my -m32 option) and get the following (snipped the output):

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libstdc++.a when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/32/libstdc++.a when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libstdc++.so when searching for -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/libstdc++.a when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libm.so when searching for -lm
/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libm.a when searching for -lm
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libm.so when searching for -lm
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libm.a when searching for -lm
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libm.so when searching for -lm
/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libm.a when searching for -lm
/usr/bin/ld: cannot find -lm

I get the same results with gcc 4.7 and gcc 4.8. The above output is from gcc 4.8.

I have gcc-4.8-multilib and g++-4.8-multilib installed.

My library path is:

LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/lib/gcc/x86_64-linux-gnu/4.8/32:/usr/lib/gcc/x86_64-linux-gnu/4.8/

From .bashrc I've specified it like this (below), in despair adding the /4.8/ and /4.8/32/ stuff after reading that the linker would only bind with the libraries that work.

export LIBRARY_PATH=/usr/lib/$(gcc -print-multiarch):/usr/lib/gcc/x86_64-linux-gnu/4.8/32:/usr/lib/gcc/x86_64-linux-gnu/4.8/

As I mention, this works quite well already on Windows as a dll, I have to believe I'm just missing something. Pointers should be 32 bits and longs should be 32 bits, the whole thing should run on x86_64. -mx32 says it will do exactly that (right?).

Checking one of the objects after compiling in -m32:

$:~/project$ file foo.o
foo.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped

And checking the same object after compiling with -mx32:

$:~/project$ file foo.o
foo.o: ELF 32-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

Am I going about this the wrong way? Can I still use my 32-bit shared library with some sort of compatibility layer?

I saw a bug report against gcc 4.7 regarding these link errors... But I didn't see much conclusion from it. This page says that gcc 4.8 is the recommended minimum for x32, so I installed it. Still -- I can't get it to link. -m32 links fine.

Answer 1

-m32 is normal 32-bit mode, and yes most x86-64 GNU/Linux systems have multiarch support with 32-bit libraries that you can install if not already installed.

The -mx32 option is not what you want at all. It compiles code for the x32 ABI, which uses 32-bit addresses with 64-bit registers and instructions. The resulting code is not compatible with 32-bit processors (as it uses 64-bit instructions), but is also not compatible with 64-bit code (which will use 64-bit addresses). So everything in user-space has to be compiled with it in mind, including libc, and even the kernel needs to support x32 user-space and syscalls: https://unix.stackexchange.com/questions/121424/linux-and-x32-abi-how-to-use (i.e. just like supporting 32-bit binaries, x32 is effectively a separate architecture.)

Complicating matters further, the testing you've been doing on Windows hasn't been doing what you think at all. Passing /DWIN32 to the Visual C++ compiler only defines a macro called WIN32 (like -DWIN32 would for GCC); it doesn't make the compiler generate 32-bit binaries. 64-bit Windows executables cannot load 32-bit libraries; the libraries you've been testing with have actually been 64-bit.

If you want to unit-test 32-bit code, you'll need to test it on a fully 32-bit target architecture like -m32. x32 is not such a system. (32-bit code under a 64-bit kernel is fine, though, unless you're worried about only having 2G or 3GiB of address-space usable by user-space in your embedded system, instead of 4GiB under a 64-bit kernel.)

Answer 2

Your initial error, trying to load the library you successfully built, indicates that you are NOT using a 32-bit version of Python, so it can't build the 32-bit shared library.

If you install an i386 (32-bit) version of Python it should be able to load the library. What's more, that is the version of Python you'll need to run on your embedded i386 target.

Note that all the above assumes your "32bit embedded processor" is i386 compatible. If its not (its an ARM or MIPS or some other 32bit embedded processor instead), you'll need to install cross compilers for that target to build your library and an emulator (such as QEMU) to run executables.