Cannot run a "docker compose up"

Question

I have been trying to work on creating my Dockerfile in which Jupyter Lab, along with the pip extensions described in the Pipfile and Pipfile.lock, runs in my Pipenv virtual environment.

Following is my Dockerfile:

FROM python:3.11 AS base

# Setup env
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONFAULTHANDLER 1

# Install pipenv and compilation dependencies
RUN apt-get update && apt-get install -y --no-install-recommends gcc
RUN pip install --upgrade pip
RUN pip install pipenv

# Copy Pipfile and Pipfile.lock
COPY Pipfile Pipfile.lock /app/

# Set up work directory
WORKDIR /app

# Install dependencies
RUN pipenv install --deploy

# Install Jupyter
RUN pipenv install jupyter notebook jupyterlab voila

# Expose ports
EXPOSE 8888
EXPOSE 5678
EXPOSE 8080
EXPOSE 80

# Create a directory for notebooks
RUN mkdir /notebooks

# Specify the directory as a volume
VOLUME /notebooks

# Create non-root user for security
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app

# Change ownership of the notebooks directory to appuser
RUN chown -R appuser:appuser /notebooks

# Switch to appuser
USER appuser

# Change permissions of the directories to ensure appuser can write to them
RUN chmod -R 755 /app && chmod -R 755 /notebooks

# Start JupyterLab
CMD ["/app/.venv/bin/pipenv", "run", "jupyter", "lab", "--notebook-dir=/notebooks", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

This is my docker-compose.yaml:

version: '3.8'
services:
  webapp:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8888:8888"
    volumes:
      - ./app:/app
      - ./notebooks:/notebooks
    user: appuser

And this is my docker-compose.debug.yaml:

version: '3.8'

services:
  webapp:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8888:8888"
      - "5678:5678"
      - "8080:8080"
    volumes:
      - ./notebooks:/notebooks
      - ./app:/app
      - ./logs:/logs
    user: appuser
    environment:
      - DEBUG=1
      - JUPYTER_PORT=8888
    command: ["/venv/bin/pipenv", "run", "jupyter", "lab", "--notebook-dir=/notebooks", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

Both Pipfile and Pipfile.lock are in the same directory along with the Dockerfile, docker-compose.yaml, and docker-compose.debug.yaml files.

When I typed sudo docker compose, it worked; however, it gave an error when I ran sudo docker compose up as follwoing:

[+] Running 1/0
 ✔ Container docker-webapp-1  Recreated                                                                          0.1s 
Attaching to webapp-1
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/app/.venv/bin/pipenv": stat /app/.venv/bin/pipenv: no such file or directory: unknown

I don't see any issues in these files, but I'm unable to start my Dockerfile. Can someone help me resolve these problems, please?

Answer 1

Your volumes: block overwrites the virtual environment in the image. You need to delete the volume hiding /app.

volumes:
  # - ./app:/app            # DELETE -- causes problems
  - ./notebooks:/notebooks  # keep this one

Your Dockerfile uses pipenv to create a virtual environment in /app/.venv. The volume mount hides everything in /app, including the virtual environment, and replaces it with content from the host.