How can I install pip-accel on Ubuntu 14.10?

Question

I would like to use pip-accel to speed up compilation-heavy installations to virtual environments (e.g. matplotlib, NumPy). It isn't in the 14.10 Ubuntu repos so I must install it manually. However, my installation attempt fails as follows:

> pip install --user pip-accel
Downloading/unpacking pip-accel
  Using download cache from /home/username/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpip-accel%2Fpip-accel-0.22.2.tar.gz
  Running setup.py (path:/tmp/pip_build_username/pip-accel/setup.py) egg_info for package pip-accel

Downloading/unpacking cached-property>=0.1.5 (from pip-accel)
  Using download cache from /home/username/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2F3.3%2Fc%2Fcached-property%2Fcached_property-0.1.5-py2.py3-none-any.whl
Downloading/unpacking coloredlogs>=0.8 (from pip-accel)
  Using download cache from /home/username/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fc%2Fcoloredlogs%2Fcoloredlogs-0.8.tar.gz
  Running setup.py (path:/tmp/pip_build_username/coloredlogs/setup.py) egg_info for package coloredlogs

Downloading/unpacking humanfriendly>=1.14 (from pip-accel)
  Using download cache from /home/username/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fh%2Fhumanfriendly%2Fhumanfriendly-1.14.tar.gz
  Running setup.py (path:/tmp/pip_build_username/humanfriendly/setup.py) egg_info for package humanfriendly

Downloading/unpacking pip>=1.4,<1.5 (from pip-accel)
  Using download cache from /home/username/.cache/pip/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpip%2Fpip-1.4.1.tar.gz
  Running setup.py (path:/tmp/pip_build_username/pip/setup.py) egg_info for package pip

    warning: no files found matching '*.html' under directory 'docs'
    warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
    no previously-included directories found matching 'docs/_build/_sources'
Installing collected packages: pip-accel, cached-property, coloredlogs, humanfriendly, pip
  Running setup.py install for pip-accel
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip_build_username/pip-accel/setup.py", line 55, in <module>
        test_suite='pip_accel.tests')
      File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python2.7/distutils/command/install.py", line 613, in run
        self.run_command(cmd_name)
      File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python2.7/distutils/dist.py", line 970, in run_command
        cmd_obj = self.get_command_obj(command)
      File "/usr/lib/python2.7/distutils/dist.py", line 845, in get_command_obj
        klass = self.get_command_class(command)
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 388, in get_command_class
        self.cmdclass[command] = cmdclass = ep.load()
      File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 2048, in load
        entry = __import__(self.module_name, globals(),globals(), ['__name__'])
      File "/usr/lib/python2.7/dist-packages/setuptools/command/install_egg_info.py", line 5, in <module>
        from setuptools.archive_util import unpack_archive
      File "/usr/lib/python2.7/dist-packages/setuptools/archive_util.py", line 15, in <module>
        from pkg_resources import ensure_directory, ContextualZipFile
    ImportError: cannot import name ContextualZipFile
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_username/pip-accel/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-kD1BYu-record/install-record.txt --single-version-externally-managed --compile --user:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/utils.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/exceptions.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/config.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/req.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/compat.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/__init__.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/tests.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/bdist.py -> build/lib.linux-x86_64-2.7/pip_accel

copying pip_accel/cli.py -> build/lib.linux-x86_64-2.7/pip_accel

creating build/lib.linux-x86_64-2.7/pip_accel/caches

copying pip_accel/caches/__init__.py -> build/lib.linux-x86_64-2.7/pip_accel/caches

copying pip_accel/caches/local.py -> build/lib.linux-x86_64-2.7/pip_accel/caches

copying pip_accel/caches/s3.py -> build/lib.linux-x86_64-2.7/pip_accel/caches

creating build/lib.linux-x86_64-2.7/pip_accel/deps

copying pip_accel/deps/__init__.py -> build/lib.linux-x86_64-2.7/pip_accel/deps

copying pip_accel/deps/debian.ini -> build/lib.linux-x86_64-2.7/pip_accel/deps

running install_lib

copying build/lib.linux-x86_64-2.7/pip_accel/utils.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/exceptions.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/config.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/req.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/compat.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/__init__.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/tests.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/caches/__init__.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel/caches

copying build/lib.linux-x86_64-2.7/pip_accel/caches/local.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel/caches

copying build/lib.linux-x86_64-2.7/pip_accel/caches/s3.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel/caches

copying build/lib.linux-x86_64-2.7/pip_accel/deps/debian.ini -> /home/username/.local/lib/python2.7/site-packages/pip_accel/deps

copying build/lib.linux-x86_64-2.7/pip_accel/deps/__init__.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel/deps

copying build/lib.linux-x86_64-2.7/pip_accel/bdist.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

copying build/lib.linux-x86_64-2.7/pip_accel/cli.py -> /home/username/.local/lib/python2.7/site-packages/pip_accel

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/utils.py to utils.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/exceptions.py to exceptions.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/config.py to config.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/req.py to req.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/compat.py to compat.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/__init__.py to __init__.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/tests.py to tests.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/caches/__init__.py to __init__.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/caches/local.py to local.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/caches/s3.py to s3.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/deps/__init__.py to __init__.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/bdist.py to bdist.pyc

byte-compiling /home/username/.local/lib/python2.7/site-packages/pip_accel/cli.py to cli.pyc

running install_egg_info

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/tmp/pip_build_username/pip-accel/setup.py", line 55, in <module>

    test_suite='pip_accel.tests')

  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup

    dist.run_commands()

  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands

    self.run_command(cmd)

  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command

    cmd_obj.run()

  File "/usr/lib/python2.7/dist-packages/setuptools/command/install.py", line 61, in run

    return orig.install.run(self)

  File "/usr/lib/python2.7/distutils/command/install.py", line 613, in run

    self.run_command(cmd_name)

  File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command

    self.distribution.run_command(command)

  File "/usr/lib/python2.7/distutils/dist.py", line 970, in run_command

    cmd_obj = self.get_command_obj(command)

  File "/usr/lib/python2.7/distutils/dist.py", line 845, in get_command_obj

    klass = self.get_command_class(command)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 388, in get_command_class

    self.cmdclass[command] = cmdclass = ep.load()

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources.py", line 2048, in load

    entry = __import__(self.module_name, globals(),globals(), ['__name__'])

  File "/usr/lib/python2.7/dist-packages/setuptools/command/install_egg_info.py", line 5, in <module>

    from setuptools.archive_util import unpack_archive

  File "/usr/lib/python2.7/dist-packages/setuptools/archive_util.py", line 15, in <module>

    from pkg_resources import ensure_directory, ContextualZipFile

ImportError: cannot import name ContextualZipFile

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_username/pip-accel/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-kD1BYu-record/install-record.txt --single-version-externally-managed --compile --user failed with error code 1 in /tmp/pip_build_username/pip-accel
Storing debug log for failure in /home/username/.pip/pip.log

Notes:

• I am using pip from the Ubuntu repos (pip 1.5.6 from /usr/lib/python2.7/dist-packages (python 2.7)) but it appears to be downloading it again (because it doesn't like 1.5.x?)
• if there's another way to use pip-accel (e.g. install it into the target virtualenv) I'm all ears, but it doesn't seem like it

Answer 1

I'm the author of pip-accel. I don't have an Ubuntu 14.10 system available for testing and on the Ubuntu 12.04 and 14.04 systems where I tested this everything works fine. However I can provide you with some hints about how to resolve this problem.

Root cause analysis

The pathnames in the traceback you included seem to hint at the root cause of this issue:

• /usr/lib/python2.7/dist-packages/setuptools is setuptools installed in /usr/lib/python2.7 (most likely installed using apt system packages)
• /usr/local/lib/python2.7/dist-packages/pkg_resources.py is pkg_resources installed in /usr/local/lib/python2.7 (most likely installed using sudo easy_install setuptools or sudo pip setuptools)

I'm pretty sure this explains why setuptools (a package that provides multiple modules) and pkg_resources (a module provided by setuptools) are "out of sync". Solving the issue might be as simple as getting rid of /usr/local/lib/python2.7/dist-packages/pkg_resources.py, however I can't tell (without more context) how that file got there and what the cleanest way to remove it is. You can try sudo pip uninstall setuptools, but please double check to make sure it will only touch files in /usr/local (you don't want pip to break the version of setuptools that's installed using apt system packages).

pip-accel 0.22 doesn't like pip 1.5.x

You mentioned:

I am using pip from the Ubuntu repos (pip 1.5.6 from /usr/lib/python2.7/dist-packages (python 2.7)) but it appears to be downloading it again (because it doesn't like 1.5.x?)

This is correct, pip-accel 0.22 required a version of pip older than 1.5. Today I released pip-accel 0.23 which uses a newer version of pip (it requires pip 6, which is what pip 1.6 would have been if pip din't change its versioning scheme to make the minor numbers the major numbers, but I digress).

pip-accel and virtual environments

You also mentioned:

if there's another way to use pip-accel (e.g. install it into the target virtualenv) I'm all ears, but it doesn't seem like it

This surprises me because the main use case of pip-accel (as documented in its README) is to install it inside the Python virtual environment where you want to install Python packages using pip-accel. So this should work fine, please try it out!