Why I get No module named 'requests' within the github actions runner?

Question

I have a gihub-actions file with a part to run pytest via poetry:

  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10',]
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install poetry
          pip install requests
          if [ -f pyproject.toml ]; then poetry install; fi
      - name: Test with pytest
        run: |
          poetry run pytest -s

However, despite the fact that I've installed the dependencies via poetry as well as requests separately via pip still it throws the following error in github-actions runner:

Run poetry run pytest -s
  
ImportError while loading conftest '/home/runner/work/backend-holoplot/backend-holoplot/tests/conftest.py'.
tests/conftest.py:4: in <module>
    from .client import override_get_db, engine
tests/client.py:3: in <module>
    from fastapi.testclient import TestClient
../../../.cache/pypoetry/virtualenvs/production-api-4u0DyUfz-py3.10/lib/python3.10/site-packages/fastapi/testclient.py:1: in <module>
    from starlette.testclient import TestClient as TestClient  # noqa
../../../.cache/pypoetry/virtualenvs/production-api-4u0DyUfz-py3.10/lib/python3.10/site-packages/starlette/testclient.py:16: in <module>
    import requests
E   ModuleNotFoundError: No module named 'requests'
Error: Process completed with exit code 4.

Answer 1

Personally solved the issue by adding requests via poetry as well instead of pip within the Installation depenecies section in github-actions as follows:

- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install poetry
    poetry install --no-interaction --all-extras
    poetry add requests

Answer 2

Usually, Poetry will install things in a separate environment. So if you install something with pip later the poetry run ... environment will not see those dependencies.

You can install requests in poetry, or you could configure poetry to not create a virtual environment in CI

for example:

name: 'tests'
on: [push, pull_request]

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          pip install poetry==1.2.2
          poetry install --no-interaction --no-ansi
      - name: Unit tests
        run: |
          poetry run pytest -vv

Will run on poetry, but you could configure to not use a virtual environment with:

name: 'tests'
on: [push, pull_request]

jobs:
  tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          pip install poetry==1.2.2
          poetry config virtualenvs.create false  # changes here
          poetry install --no-interaction --no-ansi
      - name: Unit tests
        run: |
          pytest -vv

Answer 3

try this:

build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10',]
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install poetry
          if [ -f pyproject.toml ]; then poetry install; fi
          poetry run pip install requests
      - name: Test with pytest
        run: |
          poetry run pytest -s