python does not find module installed with pipx

Question

Debain stable wants me to install Python modules using pipx. So I do

$ pipx install auditwheel
$ pipx ensurepath
$ python3 -m pipx ensurepath
$ python3
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import auditwheel
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'auditwheel'
>>> 

What am I doing wrong?

Answer 1

From Python 3.11 onward, Debian encourages the users to create a separate Python virtual environment to install Python packages.

Because Debian declares its Python install to be externally-managed, pip (and other installers) will refuse to install packages system-wide. Installation is only possible in virtual environments or separate Python installs. This is because Python package installers (like pip) are unaware of the constraints that APT-managed packages have on libraries and versions. See PEP-668 for a full discussion of the problems that can occur when multiple installers operate on the same Python install.

Therefore, the optimal way is to create a virtual environment, say MyEnv, and install packages therein:

$ mkdir -p $HOME/.venvs  # create a folder for all virtual environments 
$ python3 -m venv $HOME/.venvs/MyEnv  # create MyEnv

This will create a directory $HOME/.venvs/MyEnv with a configuration file pyvenv.cfg which includes some details for this virtual environment, such as the Python executable and Python version.

Verify the version of the Python in the virtual environment:

$HOME/.venvs/MyEnv/bin/python --version

The executables of the created virtual environment are found under $HOME/.venvs/MyEnv/bin.

To install a package into the virtual environment, use

$HOME/.venvs/MyEnv/bin/python -m pip install <some-package>

To 'activate' the virtual enviroment, i.e. adding its configuration variables into the shell environment, use

source $HOME/.venvs/MyEnv/bin/activate

Consult Python's guide to virtualenv and pip at https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments.

Answer 2

pipx is not exactly the same as pip. pipx installs applications in an isolated environment. It will not help if you want to compile and import a module. You can use a virtual environment as @AlQuemist suggests.

Answer 3

Debian docs at https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#python3-pep-668

If you need to install a Python application (or version) that isn't packaged in Debian, we recommend that you install it with pipx (in the pipx Debian package). pipx will set up an environment isolated from other applications and system Python modules, and install the application and its dependencies into that.

Emphasis mine, the docs do talk about application and under that paragraph is second one pointing to installing packages to the virtual environment;

If you need to install a Python library module (or version) that isn't packaged in Debian, we recommend installing it into a virtualenv, where possible. You can create virtualenvs with the venv Python stdlib module (in the python3-venv Debian package) or the virtualenv Python 3rd-party tool (in the virtualenv Debian package). For example, instead of running pip install --user foo, run: mkdir -p ~/.venvs && python3 -m venv ~/.venvs/foo && ~/.venvs/foo/bin/python -m pip install foo to install it in a dedicated virtualenv.

For which @AlQuemist's answer also highlights ..