How to load to same library twice from different path in same process?

Question

I am writing a new program, that is encrypting data using OpenSSL. I want my program to be FIPS compliant. To enforce FIPS mode, I am setting it using API FIPS_mode_set. I also need to transport the encrypted data to a third-party device.

The device vendor has also provided a library to communicate with the device. That library also has a dependency on the OpenSSL library.

The OpenSSL library on which my program is directly dependent on, is residing on a different path than the one that the third-party library dependent on.

The load sequence of libraries is, along with my program, the direct dependent OpenSSL library is loaded. And at runtime, I am loading the third-party library to communicate with the device. As a side effect, it should again try to load the OpenSSL library from the different path.

In case of windows both the libraries from different path are loaded in the process, and working independently. So the program is working smoothly on windows.

However, in case of Linux, only a single OpenSSL library is loaded. And I am setting it into FIPS mode. And when the third-party library tries to call a API sha1_init(), which is non FIPS call, the program is crashing.

Now, I am wondering if there is a way to load the two OpenSSL libraries independently in Linux as they are loaded in windows?

If no, than what would you suggest a way forward from this situation?

Answer 1

I am wondering if there is a way to load the two OpenSSL libraries independently in Linux as they are loaded in windows?

UNIX (and Linux) use a very different shared library model. On Windows, each DLL is in its own world, is a self-contained unit, calls its own malloc, etc.

On UNIX, everything is in a single shared namespace, symbols are shared, etc.

Even if you managed to load two separate libopenssl.so from different paths, their symbols will still be in a single namespace, and the first library to be loaded will "win" (will get its symbols used, regardless of whether the call comes from your program, or from the vendor library). This is by design.

Now, you can use dlmopen (a GLIBC extension) to have completely separate linker "domains". That would give you essentially a Windows DLL model.

However, using dlmopen has various complicated requirements (libraries must have all of their dependencies correctly specified), and introduces other complications (for example, you will now have two separate malloc implementations in your process space, and you must not mix them up).

A better approach might be to use two separate processes -- one that has most of your code and is FIPS-compliant, and a second which links your vendor library into a server to which your main program sends requests. There is obviously some overhead due to the IPC which is required in this setup, but it also isolates you from bugs in the vendor library.