Background
We have project with the following high-level directory structure*
./datascience/
├── core
│ └── setup.py
├── notebooks
│ └── Pipfile
└── web
└── Pipfile
*Excluded all the irrelevant files and directories for brevity.
The core
package is a library. It's a dependency of both the notebooks
and web
applications.
The core
package, being a library, has its dependencies specified in setup.py
import setuptools
setuptools.setup(
install_requires=[
'some-dependency',
'another-dependency'
]
)
The web
and notebooks
applications are using pipenv for dependency management. Their dependencies are specified in a Pipfile
.
For example, here's how the web
dependencies are specified in web/Pipfile
:
[packages]
datascience-core = {path = "./../core"}
flask = "~= 1.0"
Notice how the core
dependency is a local dependency, hence the relative path.
Problem
Doing a pipenv install
from inside the the web
or notebooks
directory, does not install the dependencies of the core
library as I expected!
I also tried using a Pipfile
for core
, hoping that pipenv would pick it up in its graph and download all the nested dependencies. But it doesn't.
How can dependencies of the core
app be installed automatically when pipenv is installing dependencies for the web
or notebooks
app?
Found a solution thanks to this comment in a pipenv issue thread: https://github.com/pypa/pipenv/issues/209#issuecomment-337409290
I've continued listing the
core
's dependencies insetup.py
.I've changed the
web
andnotebook
apps to use an editable installation of thecore
package.This was done by running the following in both the
web
andnotebooks
directory:It produced this diff
Now running
pipenv install
from theweb
andnotebooks
directory results in the installation of thecore
package and its dependencies!It also solved another very annoying problem, which was having to
pipenv install
every time there was a change incore
. Now it picks up development changes without having to re-install the local package!