Installing from github (with extras) via a requirements.txt file

Question

I am trying to add the Haystack library as a dependency of a python project. The installation instructions that Haystack provides are as follows

git clone https://github.com/deepset-ai/haystack.git
cd haystack
pip install -e .[all]

I am trying to translate this into a single line that I can include in a requirements.txt. My current best guess is

farm-haystack[all] @ git+https://github.com/deepset-ai/haystack.git

However this emits a bunch of warnings that various versions of farm-haystack don't provide the desired extras, such as

WARNING: farm-haystack 0.1.0.post2 does not provide the extra 'ray'

before failing with the error message

ERROR: Requested dill from https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from farm-haystack[all]@ git+https://github.com/deepset-ai/haystack.git->-r /dss_data/tmp/pip-requirements-install/req3361828774079305889.txt (line 1)) has different version in metadata: '0.3.1.dev0'

What is the proper way to go about doing this?

Answer 1

Since you mention it would be a dependency of another project, the format for listing Haystack in your requirements.txt should be the following (I pin the version here but it's not mandatory):

farm-haystack[all]==1.5.0

If you want to pin a specific git commit instead, the line in your requirements.txt should just be:

git+https://github.com/deepset-ai/haystack.git#egg=farm-haystack[all]