I'm encountering issues while trying to build my Python app on the Heroku-22 stack. I'm using scikit-learn==0.23 as a dependency, and the build process is failing with various Cython compilation errors.Here's the error log I'm receiving:
Building on the Heroku-22 stack
Using buildpack: heroku/python
Python app detected
No Python version was specified. Using the buildpack default: python-3.11.6
To use a different version, see: https://devcenter.heroku.com/articles/python-runtimes
"getting dependencies"
Installing python-3.11.6
Installing pip 23.3.1, setuptools 68.2.2 and wheel 0.42.0
Installing SQLite3
Installing requirements with pip
Collecting absl-py==1.4.0 (from -r requirements.txt (line 1))
Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)
Collecting asgiref==3.7.2 (from -r requirements.txt (line 2))
Downloading asgiref-3.7.2-py3-none-any.whl.metadata (9.2 kB)
Collecting asttokens==2.0.8 (from -r requirements.txt (line 3))
Downloading asttokens-2.0.8-py2.py3-none-any.whl (23 kB)
Collecting astunparse==1.6.3 (from -r requirements.txt (line 4))
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting Babel==2.11.0 (from -r requirements.txt (line 5))
Downloading Babel-2.11.0-py3-none-any.whl (9.5 MB)
Collecting backcall==0.2.0 (from -r requirements.txt (line 6))
Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting bcrypt==4.0.1 (from -r requirements.txt (line 7))
Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (593 kB)
Collecting beautifulsoup4==4.6.0 (from -r requirements.txt (line 8))
Downloading beautifulsoup4-4.6.0-py3-none-any.whl (86 kB)
Collecting bs4==0.0.1 (from -r requirements.txt (line 9))
Downloading bs4-0.0.1.tar.gz (1.1 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting cachetools==5.3.0 (from -r requirements.txt (line 10))
Downloading cachetools-5.3.0-py3-none-any.whl (9.3 kB)
Collecting certifi==2022.12.7 (from -r requirements.txt (line 11))
Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting cffi==1.15.1 (from -r requirements.txt (line 12))
Downloading cffi-1.15.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (462 kB)
Collecting chardet==3.0.4 (from -r requirements.txt (line 13))
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting charset-normalizer==3.1.0 (from -r requirements.txt (line 14))
Downloading charset_normalizer-3.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (197 kB)
Collecting click==8.1.3 (from -r requirements.txt (line 15))
Downloading click-8.1.3-py3-none-any.whl (96 kB)
Collecting colorama==0.4.5 (from -r requirements.txt (line 16))
Downloading colorama-0.4.5-py2.py3-none-any.whl (16 kB)
Collecting contourpy==1.0.5 (from -r requirements.txt (line 17))
Downloading contourpy-1.0.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (296 kB)
Collecting cryptography==38.0.1 (from -r requirements.txt (line 18))
Downloading cryptography-38.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
Collecting cycler==0.11.0 (from -r requirements.txt (line 19))
Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting debugpy==1.6.3 (from -r requirements.txt (line 20))
Downloading debugpy-1.6.3-py2.py3-none-any.whl (4.4 MB)
Collecting decorator==5.1.1 (from -r requirements.txt (line 21))
Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting python-decouple==3.8 (from -r requirements.txt (line 99))
Downloading python_decouple-3.8-py3-none-any.whl (9.9 kB)
Collecting pytz==2022.7.1 (from -r requirements.txt (line 100))
Downloading pytz-2022.7.1-py2.py3-none-any.whl (499 kB)
Collecting pyzmq==24.0.0 (from -r requirements.txt (line 102))
Downloading pyzmq-24.0.0-cp311-cp311-manylinux_2_28_x86_64.whl (1.1 MB)
Collecting regex==2022.10.31 (from -r requirements.txt (line 103))
Downloading regex-2022.10.31-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (781 kB)
Collecting requests==2.28.2 (from -r requirements.txt (line 104))
Downloading requests-2.28.2-py3-none-any.whl (62 kB)
Collecting requests-oauthlib==1.3.1 (from -r requirements.txt (line 105))
Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting rsa==4.9 (from -r requirements.txt (line 106))
Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting scikit-learn==1.1.2 (from -r requirements.txt (line 107))
Downloading scikit-learn-1.1.2.tar.gz (7.0 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
Preparing metadata (pyproject.toml) did not run successfully.
exit code: 1
[1913 lines of output]
Partial import of sklearn during the build process.
setup.py:128: DeprecationWarning:
> `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
of the deprecation of `distutils` itself. It will be removed for
Python >= 3.12. For older Python versions it will remain present.
It is recommended to use `setuptools < 60.0` for those Python versions.
For more details, see:
https://numpy.org/devdocs/reference/distutils_status_migration.html
from numpy.distutils.command.build_ext import build_ext # noqa
INFO: C compiler: gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
INFO: compile options: '-c'
INFO: gcc: test_program.c
INFO: gcc objects/test_program.o -o test_program
INFO: C compiler: gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC
INFO: compile options: '-c'
extra options: '-fopenmp'
INFO: gcc: test_program.c
INFO: gcc objects/test_program.o -o test_program -fopenmp
Compiling sklearn/__check_build/_check_build.pyx because it changed.
Compiling sklearn/preprocessing/_csr_polynomial_expansion.pyx because it changed.
Compiling sklearn/_loss/_loss.pyx because it changed.
Compiling sklearn/cluster/_dbscan_inner.pyx because it changed.
Compiling sklearn/cluster/_hierarchical_fast.pyx because it changed.
Compiling sklearn/cluster/_k_means_common.pyx because it changed.
Compiling sklearn/cluster/_k_means_lloyd.pyx because it changed.
Compiling sklearn/cluster/_k_means_elkan.pyx because it changed.
Compiling sklearn/cluster/_k_means_minibatch.pyx because it changed.
Compiling sklearn/datasets/_svmlight_format_fast.pyx because it changed.
Compiling sklearn/decomposition/_online_lda_fast.pyx because it changed.
Compiling sklearn/decomposition/_cdnmf_fast.pyx because it changed.
Compiling sklearn/ensemble/_gradient_boosting.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/_gradient_boosting.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/histogram.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/splitting.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/_binning.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/common.pyx because it changed.
Compiling sklearn/ensemble/_hist_gradient_boosting/utils.pyx because it changed.
Compiling sklearn/feature_extraction/_hashing_fast.pyx because it changed.
Compiling sklearn/manifold/_utils.pyx because it changed.
Compiling sklearn/manifold/_barnes_hut_tsne.pyx because it changed.
Compiling sklearn/metrics/cluster/_expected_mutual_info_fast.pyx because it changed.
Compiling sklearn/metrics/_pairwise_fast.pyx because it changed.
Compiling sklearn/metrics/_dist_metrics.pyx because it changed.
Compiling sklearn/metrics/_pairwise_distances_reduction.pyx because it changed.
Compiling sklearn/neighbors/_ball_tree.pyx because it changed.
Compiling sklearn/neighbors/_kd_tree.pyx because it changed.
Compiling sklearn/neighbors/_partition_nodes.pyx because it changed.
2. Use an 'int' return type on the function to allow an error code to be returned.
[54/59] Cythonizing sklearn/utils/_typedefs.pyx
[55/59] Cythonizing sklearn/utils/_vector_sentinel.pyx
[56/59] Cythonizing sklearn/utils/_weight_vector.pyx
performance hint: sklearn/utils/_weight_vector.pyx:178:29: Exception check will always require the GIL to be acquired.
"""
Possible solutions:
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: sklearn/_loss/_loss.pyx:122:5: Exception check on 'cgrad_hess_half_squared_error' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
[ 4/59] Cythonizing sklearn/cluster/_dbscan_inner.pyx
[ 5/59] Cythonizing sklearn/cluster/_hierarchical_fast.pyx
warning: sklearn/metrics/_dist_metrics.pxd:12:64: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.11/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
> ^^^^^^^^^^^^^^^^^^^
File "/app/.heroku/python/lib/python3.11/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
> ^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1345, in cythonize_one_helper
return cythonize_one(*m)
>
^^^^^^^^^^^^^^^^^
"error compling cython file"
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: sklearn/ensemble/_hist_gradient_boosting/splitting.pyx
>
1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
2. Use an 'int' return type on the function to allow an error code to be returned.
performance hint: sklearn/linear_model/_cd_fast.pyx:155:13: Exception check will always require the GIL to be acquired. """
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/app/.heroku/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.heroku/python/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 174, in prepare_metadata_for_build_wheel
self.run_setup()
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 268, in run_setup
self).run_setup(setup_script=setup_script)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 329, in <module>
setup_package()
File "setup.py", line 325, in setup_package
setup(**metadata)
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/numpy/distutils/core.py", line 135, in setup
config = configuration()
> ^^^^^^^^^^^^^^^
File "setup.py", line 187, in configuration
config.add_subpackage("sklearn")
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/numpy/distutils/misc_util.py", line 1050, in add_subpackage
config_list = self.get_subpackage(subpackage_name, subpackage_path,
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/numpy/distutils/misc_util.py", line 1016, in get_subpackage
config = self._get_configuration_from_setup_py(
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-4sc0oluj/overlay/lib/python3.11/site-packages/numpy/distutils/misc_util.py", line 958, in _get_configuration_from_setup_py
config = setup_module.configuration(*args)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-install-51aaryw2/scikit-learn_bf4fba3361504b2b9310e6d54a75868b/sklearn/setup.py", line 85, in configuration
cythonize_extensions(top_path, config)
File "/tmp/pip-install-51aaryw2/scikit-learn_bf4fba3361504b2b9310e6d54a75868b/sklearn/_build_utils/__init__.py", line 74, in cythonize_extensions
config.ext_modules = cythonize(
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
Encountered error while generating package metadata.
See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Push rejected, failed to compile Python app.
Push failed
Actions Taken:
I've attempted the following steps to resolve the issue:
Installed Cython version 0.29.36: pip install Cython==0.29.36
Tried installing scikit-learn version 0.23 without build isolation: pip install scikit-learn==0.23 --no-build-isolation