Compile and use python-openzwave with open-zwave in non-standard location

Question

I manually compiled python-openzwave to work with C++ library.

I would like to use it as Kodi addon (OpenELEC running on Pi 3), so can not use standard installation. I've compiled everything, downloaded missing six and louie libs, and now try to run hello_world.py.

My current dirs structure is the following:

- root
  - bin
      - .lib
      - config
        Alarm.o
        ...
        libopenzwave.a
        libopenzwave.so
        libopenzwave.so.1.4
        ...
  - libopenzwave
      driver.pxd
      group.pxd
      ...
  - louie
      __init__.py
      dispatcher.py
      ...
  - openzwave
      __init__.py
      command.py
      ...
  six.py
  hello_world.py

But when I run hello_world.py, I get the following error -

Traceback (most recent call last):
  File "hello_world.py", line 40, in <module> 
    from openzwave.controller import ZWaveController 
  File "/storage/.kodi/addons/service.multimedia.open-zwave/openzwave/controller.py", line 34, in <module> 
    from libopenzwave import PyStatDriver, PyControllerState 
ImportError: No module named libopenzwave

If I move libopenzwave.a and libopenzwave.so to root folder, then I get the following error:

Traceback (most recent call last):
  File "hello_world.py", line 40, in <module> 
    from openzwave.controller import ZWaveController 
  File "/storage/.kodi/addons/service.multimedia.open-zwave/openzwave/controller.py", line 34, in <module> 
    from libopenzwave import PyStatDriver, PyControllerState 
ImportError: dynamic module does not define init function (initlibopenzwave)

What is wrong with my setup?

Answer 1

In general the steps required consist of calls to make build which handles building the .cpp files for openzwave and downloading all dependencies (including Cython); and make install which runs the setup-api, setup-lib.py (this setup script also creates the C++ Python extention for openzwave), setup-web.py and setup-manager.py.

Since you cannot run make install as you specified and are instead using the archive they provide, the only other options for creating the python extention, after building the openzwave library with make build, is generating the .so files for it without installing to standard locations.

Building the .so for the cython extention in the same folder as the Cython scripts is done by running:

python setup.py build_ext --inplace

This should create a shared library in src-lib named libopenzwave.so (it is different from the libopenzwave.so contained in the bin/ directory) which contains all the functionality specified in the extention module. You could try adding that to the libopenzwave folder.

If you pass special compiler flags during make build for building the openzwave library you should specify the same ones when executing the setup-lib.py script. This can be done by specifying the CFLAGS before executing it (as specified here) or else you might have issues like error adding symbols: File in wrong format.

Answer 2

Here's the description of the python-openzwave's build from the question's perspective. Almost all the steps correspond to the root Makefile's targets.

• Prerequisites. There are several independent targets with little to no organization. Most use Debian-specific commands.
  • Cython is not needed if building from an archive (details below)
• openzwave C++ library (openzwave openzwave/.lib/ target).
  • Build logic: openzwave/Makefile, invoked without parameters (but with inherited environment).
  • Inputs: openzwave/ subtree (includes libhidapi and libtinyxml, statically linked).
  • Outputs: openzwave/.lib/libopenzwave.{a,so}
  • Accepts PREFIX as envvar (/usr/local by default)
    • The only effect that affects us is: $(PREFIX)/etc/openzwave/ is assigned to a macro which adds a search location for config files (Options.cpp): config/ -> /etc/openzwave/ -> <custom location>.
• libopenzwave Python C extension module (install-lib target - yes, the stock Makefile cannot just build it; the target doesn't even have the dependency on the library).
  • Build logic: setup-lib.py
  • Inputs: src-lib/, openzwave/.lib/libopenzwave.a
  • Outputs: build/<...>/libopenzwave.so - yes, the same name as openzwave's output, so avoid confusing them
    • By default, openzwave is linked statically with the module so you don't need to include the former into a deployment
    • The module does, however, need the config folder from the library. It is included by the build script when making a package.
  • Contrary to what Jim says, Cython is not needed to build from an archive, the archive already includes the generated .cpp.
  • Now, the catch is: the module itself uses pkg_resources to locate its data. So you cannot just drop the .so and config into the currect directory and call it a day. You need to make pkg_resources.get_distribution('libopenzwave') succeed.
    • pkg_resources claims to support "normal filesystem packages, .egg files, and unpacked .egg files."
    • In particular, I was able to pull this off: make an .egg (setup-lib.py bdist_egg), unpack it into the current directory and rename EGG-INFO into libopenzwave.egg-info (like it is in site-packages). A UserWarning is issued if I don't specifically add the .so's location into PYTHON_PATH/sys.path before importing the module.
• openzwave,pyozwman and pyozwweb Python packages (install)
  • these are pure Python packages. The first one uses the C extension module, others use the first one.
  • Build logic: setup-api.py,setup-manager.py,setup-web.py
  • Input: src-*/
  • Output: (pure Python)
  • They only use pkg_resources.declare_namespace() so you're gonna be fine with just the proper files/dirs on sys.path without any .egg-info's