Installing QScintilla2 for Anaconda on OSX: an @rpath issue

Question

With a lot of blood, sweat and tears I have managed to compile QScintilla2 (https://www.riverbankcomputing.com/software/qscintilla/download) for usage in the Anaconda Python Distribution (2.5.0; Python 2.7.11; PyQt4) on Mac OS X El Capitan.

After compilation and installation of everything inside the Qt4Qt5 and Python folders without errors or warnings, all appears to have ended up in the correct place in Anaconda.

However, when I try to import qscintilla2 with

import PyQt4.Qsci

I get the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/daniel/anaconda/lib/python2.7/site-packages/PyQt4/Qsci.so, 2): Library not loaded: @rpath/./libQtGui.4.dylib
  Referenced from: /Users/daniel/anaconda/lib/python2.7/site-packages/PyQt4/Qsci.so
  Reason: image not found

Thus the module does not seem to be able to find libQtGui.4.dylib

I have done some reading up on the @rpath variable on the excellent blog https://mikeash.com/pyblog/friday-qa-2009-11-06-linking-and-install-names.html and ended up with knowing I have to use otool and install_name_tool in my attempts to solve this problem.

If I run otool -L I get the following output:

Qsci.so:
    libQsci.dylib (compatibility version 0.0.0, current version 0.0.0)
    /Users/daniel/anaconda/lib/libqscintilla2.11.dylib (compatibility version 11.3.0, current version 11.3.0)
    @rpath/./libQtGui.4.dylib (compatibility version 4.8.0, current version 4.8.7)
    @rpath/./libQtCore.4.dylib (compatibility version 4.8.0, current version 4.8.7)
    /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 104.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

and otool -l gives me

Qsci.so:
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
   vmaddr 0x0000000000000000
   vmsize 0x000000000009b000
  fileoff 0
 filesize 634880
  maxprot 0x00000007
 initprot 0x00000005
   nsects 7
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x0000000000002450
      size 0x000000000006bcf6
    offset 9296
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x000000000006e146
      size 0x0000000000001956
    offset 450886
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x000000000006fa9c
      size 0x00000000000032ac
    offset 457372
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000000072d50
      size 0x0000000000008e47
    offset 470352
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x000000000007bb97
      size 0x0000000000000f62
    offset 506775
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __unwind_info
   segname __TEXT
      addr 0x000000000007cafc
      size 0x00000000000014b4
    offset 510716
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __eh_frame
   segname __TEXT
      addr 0x000000000007dfb0
      size 0x000000000001d048
    offset 516016
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __DATA
   vmaddr 0x000000000009b000
   vmsize 0x0000000000027000
  fileoff 634880
 filesize 159744
  maxprot 0x00000007
 initprot 0x00000003
   nsects 7
    flags 0x0
Section
  sectname __dyld
   segname __DATA
      addr 0x000000000009b000
      size 0x0000000000000010
    offset 634880
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __got
   segname __DATA
      addr 0x000000000009b010
      size 0x0000000000000030
    offset 634896
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 1081 (index into indirect symbol table)
 reserved2 0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x000000000009b040
      size 0x00000000000021c8
    offset 634944
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 1087 (index into indirect symbol table)
 reserved2 0
Section
  sectname __const
   segname __DATA
      addr 0x000000000009d210
      size 0x00000000000050f8
    offset 643600
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __data
   segname __DATA
      addr 0x00000000000a2310
      size 0x000000000001f091
    offset 664336
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __bss
   segname __DATA
      addr 0x00000000000c13b0
      size 0x0000000000000430
    offset 0
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Section
  sectname __common
   segname __DATA
      addr 0x00000000000c17e0
      size 0x0000000000000030
    offset 0
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000001
 reserved1 0
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x00000000000c2000
   vmsize 0x0000000000063000
  fileoff 794624
 filesize 402904
  maxprot 0x00000007
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 40
         name libQsci.dylib (offset 24)
   time stamp 1 Thu Jan  1 01:00:01 1970
      current version 0.0.0
compatibility version 0.0.0
Load command 4
     cmd LC_SYMTAB
 cmdsize 24
  symoff 863408
   nsyms 5494
  stroff 964424
 strsize 233104
Load command 5
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 2227
     iextdefsym 2227
     nextdefsym 2034
      iundefsym 4261
      nundefsym 1233
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 955752
  nindirectsyms 2168
      extreloff 951312
        nextrel 555
      locreloff 794624
        nlocrel 7955
Load command 6
     cmd LC_UUID
 cmdsize 24
    uuid 888BE029-40E0-3D69-83D4-B236B57ADFD4
Load command 7
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.5
      sdk 10.11
Load command 8
          cmd LC_LOAD_DYLIB
      cmdsize 80
         name /Users/daniel/anaconda/lib/libqscintilla2.11.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 11.3.0
compatibility version 11.3.0
Load command 9
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/./libQtGui.4.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 4.8.7
compatibility version 4.8.0
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name @rpath/./libQtCore.4.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 4.8.7
compatibility version 4.8.0
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libstdc++.6.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 104.1.0
compatibility version 7.0.0
Load command 12
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 1226.10.1
compatibility version 1.0.0
Load command 13
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 858264
 datasize 5144
Load command 14
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 863408
 datasize 0

As far as I can see it, there is no entry of @rpath present in this module (or at least I should have found an LC_RPATH command?). I can set this of course with install_name_tool afterwards, but that's where I get stuck.

I don't understand what in this situation is regarded as @loader_path and @executable_path for the Qsci.so module. It is located at

/Users/daniel/anaconda/lib/python2.7/site-packages/PyQt4/Qsci.so

but it seems to be a direct reference to

/Users/daniel/anaconda/lib/libqscintilla2.11.dylib

So would /Users/daniel/anaconda/lib/python2.7/site-packages/PyQt4/ or /Users/daniel/anaconda/lib/ be the @loader_path in this case? I assume that the @executable_path just points to the path of the Python interpreter which is in /Users/daniel/anaconda/bin, but correct me if I am wrong.

Both libQtGui.4.dylib and libQtCore.4.dylib are located at

/Users/daniel/anaconda/lib/

I tried to add what I think are the correct @rpaths manually with

install_name_tool -add_rpath @loader_path/../../ Qsci.so
install_name_tool -add_rpath @loader_path/../lib Qsci.so

the first entry assuming the containing folder of Qsci.so to be the @loader_path and the second one of libqscintilla2.11.dylib, but to no avail...

If I add the absolute path to @rpath with

install_name_tool -add_rpath /Users/daniel/anaconda/lib Qsci.so

it does solve the problem, so I do know this is an rpath issue. I do need the relative locations however as I plan to package the app once I get this right.

Thanks for any help with this, and my apologies that this post has become way longer than intended!

Answer 1

By inspecting other PyQt4 modules, I figured out I had to add an rpath entry with install_name_tool pointing to @loader_path/../../../ in Qsci.so. Below you'll find a bash script which goes through the whole installation process of QScintilla2 for Anaconda on a Mac.

#!/bin/bash
# Run this file from within the root of the QScintilla source folder

# Go to Qscintilla source dir
cd Qt4Qt5
# Build the makefile with qmake, specify llvm as the compiler
# The normal g++ compiler causes an __Unwind_Resume error at linking phase
~/anaconda/bin/qmake qscintilla.pro -spec macx-llvm
# Build Qscintilla
make
# and install it
make install

# Go to python folder
cd ../Python

# Configure compilation of Python Qsci module
~/anaconda/bin/python configure.py -q ~/anaconda/bin/qmake --sip ~/anaconda/bin/sip -n ~/anaconda/include -o ~/anaconda/lib
# make it
make
# Add the correct @rpath entry to Qsci.so so it can find the other required Qt modules
install_name_tool -add_rpath @loader_path/../../../ Qsci.so
# Install QSci.so to the site-packages/PyQt4 folder
make install