Install youtokentome in poetry requires Cython, unable to configre correctly

Question

I'm trying to convert whipser-diarization to poetry

It goes well until I add nemo_toolkit[asr]==1.20.0, which depends on youtokentome (that name is well thought of, btw)

    File "/tmp/tmpexmdke23/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 507, in run_setup
      super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
    File "/tmp/tmpexmdke23/.venv/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in run_setup
      exec(code, locals())
    File "<string>", line 5, in <module>
  ModuleNotFoundError: No module named 'Cython'

So I tried adding cython to the dependencies. It works fine if I poetry shell and execute cython, so it is avaible.

My pyproject so far:

...
[tool.poetry.dependencies]
python = "^3.10"
faster-whisper = "0.9.0"
wget = "^3.2"
transformers = ">=4.26.1"
whisperx = {git = "https://github.com/m-bain/whisperX.git", rev = "49e0130e4e0c0d99d60715d76e65a71826a97109"}
deepmultilingualpunctuation = "^1.0.1"
cython = "^3.0.5"


[build-system]
requires = ["poetry-core", "cython"]
build-backend = "poetry.core.masonry.api"

I added cython to the requires, but that does nothing to prevent the error.

Any hints?

Answer 1

Building this package is done in an isolated environment, so it doesn't matter if Cython is installed in your current environment.

youtokentome has to define its build requirements according to PEP 518.

There seems to be a Pull Request for it for a long time: https://github.com/VKCOM/YouTokenToMe/pull/108

Answer 2

I had to run the following (Python 3.10.12):

pip install pybind11                                                          
pip install wheel setuptools pip --upgrade
pip install fasttext
pip install git+https://github.com/gburlet/YouTokenToMe.git@dependencies  
pip install "nemo_toolkit[all]"==1.21