How to update tox environments when requirements.txt files change?

Question

Question

Normally when I change the deps in my tox.ini file tox will notice the change and recreate the virtualenv with the new dependencies. But if I use deps = -r requirements.txt to read my dependencies from a requirements.txt file then tox doesn't update the virtualenv when requirements.txt changes. How can I automatically keep my tox virtualenvs in sync with my requirements.txt files?

Details

tox detects changes to deps in tox.ini

When using the deps setting in tox.ini to list your dependencies if you change the deps then the next time you run a tox command it'll notice the change and will create the virtualenv and install the new deps into the new virtualenv. Here's a minimal tox.ini file to demonstrate what I'm talking about:

# tox.ini
[tox]
skipsdist = true

[testenv]
deps = pytest
commands = pytest --version

If you run tox in a directory containing this tox.ini file then tox will create a virtualenv in the .tox directory, install pytest into that virtualenv, and run pytest --version in the virtualenv. If you run tox again it'll reuse the existing virtualenv and just run pytest --version without unnecessarily reinstalling pytest again. If you make a change to the deps in tox.ini, for example like this:

diff --git a/tox.ini b/tox.ini
index 7d92601..e45a612 100644
--- a/tox.ini
+++ b/tox.ini
@@ -2,5 +2,5 @@
 skipsdist = true
 
 [testenv]
-deps = pytest
-commands = pytest --version
+deps = pylint
+commands = pylint --version

Then the next time I run tox it'll recreate the virtualenv and reinstall the dependencies before running the commands:

$ tox
python recreate: /tmp/tox/.tox/python
python installdeps: pylint
python installed: astroid==2.12.4,dill==0.3.5.1,isort==5.10.1,lazy-object-proxy==1.7.1,mccabe==0.7.0,platformdirs==2.5.2,pylint==2.15.0,tomli==2.0.1,tomlkit==0.11.4,wrapt==1.14.1
python run-test-pre: PYTHONHASHSEED='4045882343'
python run-test: commands[0] | pylint --version
pylint 2.15.0
astroid 2.12.4
Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
___________________________________ summary ____________________________________
  python: commands succeeded
  congratulations :)

You can put requirements files in deps

You can reference pip requirements files from tox.ini with -r, for example:

# tox.ini
[tox]
skipsdist = true

[testenv]
deps = -r requirements.txt
commands = pylint --version

Now when you run tox it'll install the dependencies from requirements.txt into the virtualenv.

The problem: tox doesn't detect changes to requirements.txt

But there's a problem: tox won't notice if the requirements.txt file changes. Your virtualenv will still contain the dependencies from the old version of the requirements.txt file. tox only notices direct changes to the deps setting in tox.ini itself.

Ideally you wouldn't want tox to recreate the virtualenv from scratch when requirements.txt changes like it does when tox.ini changes: requirements.txt files are often quite large and reinstalling them from scratch can take a long time. Ideally tox would update the virtualenv in place: installing, removing, updating and downgrading packages as necessary to synchronize the virtualenv with the requirements.txt file.

Answer 1

You can use the pip-sync command from pip-tools to keep your tox virtualenv synchronized with any changes to your requirements.txt file:

# tox.ini
[tox]
skipsdist = true

[testenv]
deps = pip-tools
commands_pre = pip-sync requirements.txt
commands = pytest --version

How this works:

1. We've removed the -r requirements.txt from the tox.ini's deps: we're no longer using tox to install our requirements.txt file.
2. Instead, we've put pip-tools in the deps. This means tox will install pip-tools (the package that contains the pip-sync command) whenever it creates a new virtualenv.
3. We've added pip-sync requirements.txt to the commands_pre setting in tox.ini. Now every time we run tox it will run pip-sync before running whatever is in the commands setting (pytest --version in this example). This pip-sync command will have no effect if the requirements.txt file hasn't changed, but if requirements.txt has changed then it'll update the virtualenv.

Speeding it up with pip-sync-faster

There's one problem: running pip-sync every time you run tox is slow, even when the requirements.txt file hasn't changed and the virtualenv doesn't need to be updated. On my machine using a large requirements.txt file from a real app it takes about 1.5s to run a simple command like pytest --version in tox. For comparison a simple tox.ini file that doesn't call pip-sync (and therefore doesn't update the virtualenv when requirements.txt changes) runs pytest --version in about 800ms.

pip-sync-faster is a pip-sync wrapper script that can speed things up by doing a very fast check of whether requirements.txt has changed and only calling pip-sync if it has.

You need to add pip-sync-faster to your requirements.txt file, otherwise it'll uninstall itself! This is because pip-sync-faster calls pip-sync which uninstalls any package that isn't in requirements.txt, including pip-sync-faster. Here's an example pinned requirements.txt file containing pip-sync-faster, pytest, and their dependencies:

attrs==22.1.0
build==0.8.0
click==8.1.3
iniconfig==1.1.1
packaging==21.3
pep517==0.13.0
pip-sync-faster==0.0.2
pip-tools==6.8.0
pluggy==1.0.0
py==1.11.0
pyparsing==3.0.9
pytest==7.1.2
tomli==2.0.1

With a requirements.txt like this you can now use pip-sync-faster with a tox.ini file like this:

# tox.ini
[tox]
skipsdist = true

[testenv]
deps = pip-sync-faster
commands_pre = pip-sync-faster requirements.txt
commands = pytest --version

With my real-world requirements.txt running pytest --version in tox now takes about 850ms, a speed up of almost 600ms.

The deps = -r requirements.txt gets tox to install pip-sync-faster whenever it creates a new virtualenv, the commands_pre = pip-sync-faster requirements.txt then installs requirements.txt into the virtualenv and updates the virtualenv if requirements.txt changes, before running the commands (remember: pip-sync-faster needs to be in requirements.txt or it'll uninstall itself!)

Speeding it up even more with tox-faster

tox-faster is a little tox plugin that can shave a few hundred more milliseconds off your tox startup time (depending on the size of your requirements.txt file). Just add it to the requires setting in your tox.ini file:

# tox.ini
[tox]
skipsdist = true
requires = tox-faster

[testenv]
deps = -r requirements.txt
commands_pre = pip-sync-faster requirements.txt
commands = pytest --version

With my app's requirements.txt file running pytest --version in tox now takes about 650ms, another 200ms faster.