I have python projects that I wish to scan using Whitesource CLI in Azure Devops Pipeline. Here are the commands that are executed. These work for most python packages in my org, but for a few, it fails.
cd $(Build.Repository.Name)
python3 -m pip install virtualenv
export ENV_NAME=$(Build.Repository.Name)_$(Build.BuildId)
python3 -m venv ${ENV_NAME}
source ${ENV_NAME}/bin/activate
python3 -m pip install wheel virtualenv
python3 -m pip install .
python3 -m pip freeze > requirements.txt
curl "http://downloads.whitesourcesoftware.com/ws-cli/master/latest/ws-cli-linux-amd64/ws" -o ws
chmod +x ./ws
./ws clear config
./ws configure --url https://app.whitesourcesoftware.com \
--api-key "$WHITESOURCE_API_KEY" \
--user-key "$WHITESOURCE_USER_KEY"
./ws scan
Here is the failure that I am unable to resolve. It seems like the file subprocess.py is getting shadowed as per some answers that I found. But I dont have control over the numpy/scipy or any other versions of any of the dependencies mentioned in different python projects scanned by the above code. How to fix?
[ERROR] package.json: /path/repo/repo_804929/lib/python3.8/site-packages/matplotlib/backends/web_backend/package.json identified without a corresponding lock file
[WARN] Errors occurred during npm resolution
[INFO] Using installed PIP version 21.1.1 (python3.8)
[DEBUG] Command - executeProcess - error in execute command 'python -m virtualenv --version', Exit Status 1
[DEBUG] Read error line #1: Traceback (most recent call last):
[DEBUG] Read error line #2: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 614, in _syscmd_uname
[DEBUG] Read error line #3: output = subprocess.check_output(('uname', option),
[DEBUG] Read error line #4: AttributeError: partially initialized module 'subprocess' has no attribute 'check_output' (most likely due to a circular import)
[DEBUG] Read error line #5: During handling of the above exception, another exception occurred:
[DEBUG] Read error line #6: Traceback (most recent call last):
[DEBUG] Read error line #7: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/__init__.py", line 144, in <module>
[DEBUG] Read error line #8: from . import core
[DEBUG] Read error line #9: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/__init__.py", line 100, in <module>
[DEBUG] Read error line #10: from . import _add_newdocs_scalars
[DEBUG] Read error line #11: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 87, in <module>
[DEBUG] Read error line #12: add_newdoc_for_scalar_type('byte', [],
[DEBUG] Read error line #13: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 62, in add_newdoc_for_scalar_type
[DEBUG] Read error line #14: alias_doc += ''.join(":Alias on this platform ({} {}): `numpy.{}`: {}.\n ".format(platform.system(), platform.machine(), alias, doc)
[DEBUG] Read error line #15: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 62, in <genexpr>
[DEBUG] Read error line #16: alias_doc += ''.join(":Alias on this platform ({} {}): `numpy.{}`: {}.\n ".format(platform.system(), platform.machine(), alias, doc)
[DEBUG] Read error line #17: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 892, in system
[DEBUG] Read error line #18: return uname().system
[DEBUG] Read error line #19: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 858, in uname
[DEBUG] Read error line #20: processor = _syscmd_uname('-p', '')
[DEBUG] Read error line #21: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 617, in _syscmd_uname
[DEBUG] Read error line #22: except (OSError, subprocess.CalledProcessError):
[DEBUG] Read error line #23: AttributeError: partially initialized module 'subprocess' has no attribute 'CalledProcessError' (most likely due to a circular import)
[DEBUG] Read error line #24: Traceback (most recent call last):
[DEBUG] Read error line #25: File "/opt/conda/miniconda/lib/python3.8/runpy.py", line 185, in _run_module_as_main
[DEBUG] Read error line #26: mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
[DEBUG] Read error line #27: File "/opt/conda/miniconda/lib/python3.8/runpy.py", line 144, in _get_module_details
[DEBUG] Read error line #28: return _get_module_details(pkg_main_name, error)
[DEBUG] Read error line #29: File "/opt/conda/miniconda/lib/python3.8/runpy.py", line 111, in _get_module_details
[DEBUG] Read error line #30: __import__(pkg_name)
[DEBUG] Read error line #31: File "/path/repo/repo_804929/lib/python3.8/site-packages/virtualenv/__init__.py", line 3, in <module>
[DEBUG] Read error line #32: from .run import cli_run, session_via_cli
[DEBUG] Read error line #33: File "/path/repo/repo_804929/lib/python3.8/site-packages/virtualenv/run/__init__.py", line 11, in <module>
[DEBUG] Read error line #34: from ..seed.wheels.periodic_update import manual_upgrade
[DEBUG] Read error line #35: File "/path/repo/repo_804929/lib/python3.8/site-packages/virtualenv/seed/wheels/__init__.py", line 3, in <module>
[DEBUG] Read error line #36: from .acquire import get_wheel, pip_wheel_env_run
[DEBUG] Read error line #37: File "/path/repo/repo_804929/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py", line 10, in <module>
[DEBUG] Read error line #38: from virtualenv.util.subprocess import Popen, subprocess
[DEBUG] Read error line #39: File "/path/repo/repo_804929/lib/python3.8/site-packages/virtualenv/util/subprocess/__init__.py", line 3, in <module>
[DEBUG] Read error line #40: import subprocess
[DEBUG] Read error line #41: File "/opt/conda/miniconda/lib/python3.8/subprocess.py", line 49, in <module>
[DEBUG] Read error line #42: import signal
[DEBUG] Read error line #43: File "/path/repo/repo_804929/lib/python3.8/site-packages/scipy/signal/__init__.py", line 309, in <module>
[DEBUG] Read error line #44: from . import _sigtools, windows
[DEBUG] Read error line #45: ImportError: numpy.core.multiarray failed to import
[DEBUG] Command - executeProcess - error in execute command 'python -m virtualenv /tmp/ws-ua_20220218111547_XQYIQF/python_OUUZFX/202202181115481/env', Exit Status 1
[DEBUG] Read error line #1: Traceback (most recent call last):
[DEBUG] Read error line #2: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 614, in _syscmd_uname
[DEBUG] Read error line #3: output = subprocess.check_output(('uname', option),
[DEBUG] Read error line #4: AttributeError: partially initialized module 'subprocess' has no attribute 'check_output' (most likely due to a circular import)
[DEBUG] Read error line #5: During handling of the above exception, another exception occurred:
[DEBUG] Read error line #6: Traceback (most recent call last):
[DEBUG] Read error line #7: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/__init__.py", line 144, in <module>
[DEBUG] Read error line #8: from . import core
[DEBUG] Read error line #9: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/__init__.py", line 100, in <module>
[DEBUG] Read error line #10: from . import _add_newdocs_scalars
[DEBUG] Read error line #11: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 87, in <module>
[DEBUG] Read error line #12: add_newdoc_for_scalar_type('byte', [],
[DEBUG] Read error line #13: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 62, in add_newdoc_for_scalar_type
[DEBUG] Read error line #14: alias_doc += ''.join(":Alias on this platform ({} {}): `numpy.{}`: {}.\n ".format(platform.system(), platform.machine(), alias, doc)
[DEBUG] Read error line #15: File "/path/repo/repo_804929/lib/python3.8/site-packages/numpy/core/_add_newdocs_scalars.py", line 62, in <genexpr>
[DEBUG] Read error line #16: alias_doc += ''.join(":Alias on this platform ({} {}): `numpy.{}`: {}.\n ".format(platform.system(), platform.machine(), alias, doc)
[DEBUG] Read error line #17: File "/opt/conda/miniconda/lib/python3.8/platform.py", line 892, in system
[DEBUG] Read error line #18: return uname().system