Optional dependencies using test-requirements.txt in pyproject.toml with setuptools

Question

A typical pyproject.toml using setuptools with optional dependencies looks like the following (unrelated sections removed):

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "my_project"
dependencies = ["numpy","pandas"]

[project.optional-dependencies]
fast = ["numba"]
test = ["pytest"]

To use a requirements.txt (generated from pip-compile using requirements.in) to store the main dependencies (without adding other files such as setup.py), one can use the dynamic keyword:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "my_project"
dynamic = ["dependencies"] # Changed

[project.optional-dependencies]
fast = ["numba"]
test = ["pytest"]

[tool.setuptools.dynamic] # New section
dependencies = {file = ["requirements.txt"]}

If I would like setuptools to also read the optional dependencies from other files (say, test-requirements.txt), what should be the correct syntax? According to the documentation, the feature is in beta, and only a single keyword optional-dependencies is exposed. But I have more than one optional dependencies: [fast] and [test], and their *-requirements.txt are generated in a fixed format from pip-compile. Specifically, what is meant by group in:

subset of the requirements.txt format per group

?

Answer 1

Found the solution from an example in pip-tools's GitHub. Adapting it to my example:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "my_project"
dynamic = ["dependencies", "optional-dependencies"]

[tool.setuptools.dynamic]
dependencies = { file = ["requirements.in"] }
optional-dependencies.fast = { file = ["fast-requirements.txt"] }
optional-dependencies.test = { file = ["test-requirements.txt"] }

I do not understand why in the example one file is .in and the others are .txt, but this will do for now. In fact, to avoid version conflicts (even with pip-sync), eventually in my repository I swapped the above example code the other way around: use the version-pinned requirements.txt, but turning the optional dependencies into .in.