pip install pylbfgs fails in a clean virtualenv

Question

On a completely fresh virtualenv, installing pylbfgs fails with the error below. My goal is to install dedupe, but it depends on pylbfgs.

I'm assuming it has something to do with the release of Cython 3.0.0 a few days ago, but even if I do pip install Cython==0.29.35 it still fails. Last commit to pylbfgs was in 2014 so something external must have changed.

Any idea how I can set up an environment to make it work?

$ pip install pylbfgs
Collecting pylbfgs
  Using cached PyLBFGS-0.2.0.14.tar.gz (98 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      /private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-install-hgmjirii/pylbfgs_27ac07c00d1741aaa6fb205fbf713b21/lbfgs/_lowlevel.pyx
        tree = Parsing.p_module(s, pxd, full_module_name)

      Error compiling Cython file:
      ------------------------------------------------------------
      ...

              x_a = aligned_copy(x0.ravel())

              try:
                  callback_data = (f, progress, x0.shape, args)
                  r = lbfgs(n, x_a, fx_final, call_eval,
                                              ^
      ------------------------------------------------------------

      lbfgs/_lowlevel.pyx:395:40: Cannot assign type 'lbfgsfloatval_t (void *, lbfgsconst_p, lbfgsfloatval_t *, int, lbfgsfloatval_t) except? -1' to 'lbfgs_evaluate_t'

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
              x_a = aligned_copy(x0.ravel())

              try:
                  callback_data = (f, progress, x0.shape, args)
                  r = lbfgs(n, x_a, fx_final, call_eval,
                            call_progress, <void *>callback_data, &self.params)
                            ^
      ------------------------------------------------------------

      lbfgs/_lowlevel.pyx:396:22: Cannot assign type 'int (void *, lbfgsconst_p, lbfgsconst_p, lbfgsfloatval_t, lbfgsfloatval_t, lbfgsfloatval_t, lbfgsfloatval_t, int, int, int) except? -1' to 'lbfgs_progress_t'
      Compiling lbfgs/_lowlevel.pyx because it changed.
      [1/1] Cythonizing lbfgs/_lowlevel.pyx
      Traceback (most recent call last):
        File "/Users/_/dev/test/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/_/dev/test/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/_/dev/test/env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 488, in run_setup
          self).run_setup(setup_script=setup_script)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 338, in run_setup
          exec(code, locals())
        File "<string>", line 27, in <module>
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
          cythonize_one(*args)
        File "/private/var/folders/jg/pb4kddl17kzf4njpdp3ht8rh0000gn/T/pip-build-env-rhor0rp4/overlay/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: lbfgs/_lowlevel.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Answer 1

Downgrading to Python <3.11 seems to work (verified with v3.10.12). Not an ideal solution though.