How to use pip-tools to install from a github repo?

Question

Here is a line in a req.in file (for pip-compile) to get a package from a github repository:

-e git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields

Running pip-compile req.in gives a req.txt with

-e git+git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields

Using pip to install (pip install -r req.txt) gives this error:

   ERROR: Command errored out with exit status 1:
     command: .../.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'.../.venv/src/django-editorjs-fields/setup.py'"'"'; __file__='"'"'.../.venv/src/django-editorjs-fields/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: .../.venv/src/django-editorjs-fields/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'

I have setuptools (and in fact all the other libraries) installed. See, no errors:

$ .venv/bin/python -c 'import io, os, sys, setuptools, tokenize;'
$

If I install directly (without the -e) it works:

$ pip install git+git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields
... (works)

But when I take the -e out of req.in, pip-compile complains:

$ pip-compile req.in
...
pip._internal.exceptions.InstallationError: Invalid requirement: 'git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields' (from line 1 of req.in)
Hint: It looks like a path. File 'git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields' does not exist.

What the heck, I'm trapped.

How do I put a github repo into req.in, compile it with pip-compile into req.txt, and pip install req.txt without errors?

I need it to work automatically because we have a toolchain that depends on it (github workflows, dependabot, ..).

I'm using python 3.9.1_2, pip 21.1.2, pip-tools 6.2.0 on OS X. Also happened with pip-tools 6.0.1 (not surprising).

Answer 1

Use the following format for git requirements:

$ cat requirements.in
django-editorjs-fields @ git+https://github.com/dfrankow/django-editorjs-fields

Result:

$ pip-compile
#
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
#    pip-compile
#
django-editorjs-fields @ git+https://github.com/dfrankow/django-editorjs-fields.git
    # via -r requirements.in