Docker, Multiple Poetry Stages, `six.moves` error

Question

I have a fastapi app that is built from two repos using Poetry. Here is an example of how it runs within the terminal. Note data_sources is a dependency for api. The api .toml references the dist/*.whl in data_sources.

    $ cd data_sources
    $ poetry install
    $ poetry build
    $ cd ..
    $ cd api
    $ poetry install
    $ poetry build
    $ uvicorn api:app

I am wanting to create a Dockerfile that produces the same procedure, but am receiving this error during docker build .

0.597   File "/opt/poetry/venv/lib/python3.12/site-packages/poetry/core/utils/_compat.py", line 8, in <module>
0.597     import six.moves.urllib.parse as urllib_parse
0.597 ModuleNotFoundError: No module named 'six.moves'
------
Dockerfile:35
--------------------
  33 |     WORKDIR /app/api
  34 |     COPY api/pyproject.toml api/poetry.lock ./
  35 | >>> RUN poetry install
  36 |     RUN poetry build
  37 |
--------------------

I have tried may ways to dockerize, but here is my latest Dockerfile.

FROM python:3.12-slim AS data_sources_builder

ENV POETRY_HOME=/opt/poetry \
    POETRY_VERSION=1.1.11 \
    PATH="/opt/poetry/bin:$PATH"

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        curl \
        build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sSL https://install.python-poetry.org | python3 -
WORKDIR /app/data_sources
COPY data_sources/pyproject.toml data_sources/poetry.lock ./
RUN poetry install
RUN poetry build


FROM python:3.12-slim AS api_builder

ENV POETRY_HOME=/opt/poetry \
    POETRY_VERSION=1.8.2 \
    PATH="/opt/poetry/bin:$PATH"

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        curl \
        build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN curl -sSL https://install.python-poetry.org | python3 -
WORKDIR /app/api
COPY api/pyproject.toml api/poetry.lock ./
RUN poetry install
RUN poetry build

FROM python:3.12-slim
COPY --from=data_sources_builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=api_builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages

WORKDIR /app/api
COPY . .
EXPOSE 8000
CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]

Answer 1

It's not clear where the dependency on six.moves is coming from. Presumably it's listed as a dependency somewhere in your project. The six package is to facilitate compatibility between Python 2 and Python 3. Where are you using it?

Assuming that your project looks something like this:

├── api
│   ├── api.py
│   ├── __init__.py
│   ├── poetry.lock
│   └── pyproject.toml
├── data_sources
│   ├── data_sources.py
│   ├── poetry.lock
│   ├── pyproject.toml
│   └── README.md
└── Dockerfile

Then this Dockerfile should work.

FROM python:3.12-slim AS builder

ENV POETRY_HOME=/opt/poetry \
    POETRY_VERSION=1.8.2 \
    PATH="/opt/poetry/bin:$PATH"

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        curl \
        build-essential \
    && rm -rf /var/lib/apt/lists/* && \
    curl -sSL https://install.python-poetry.org | python3 - && \
    poetry config virtualenvs.create false

WORKDIR /app/

# =============================================================================

FROM builder as data_sources_builder

COPY data_sources/pyproject.toml data_sources/poetry.lock .

RUN poetry install

COPY data_sources/ .

RUN poetry build

# =============================================================================

FROM builder AS api_builder

COPY api/pyproject.toml api/poetry.lock .

RUN poetry install

COPY api/ .

RUN poetry build

# =============================================================================

FROM python:3.12-slim

COPY --from=data_sources_builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=api_builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=api_builder /usr/local/bin/uvicorn /usr/local/bin/

WORKDIR /app/

COPY . .

EXPOSE 8000

CMD ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]

The pyproject.toml file in api/ has:

[tool.poetry]
name = "api"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.12"
fastapi = "^0.110.0"
uvicorn = "^0.29.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"