Problem running a Docker container in Gitlab CI/CD

Question

I am trying to build and run my Docker image using Gitlab CI/CD, but there is one issue I can't fix even though locally everything works well.

Here's my Dockerfile:

FROM <internal_docker_repo_image>
RUN apt update && \
    apt install --no-install-recommends -y build-essential gcc
COPY requirements.txt /requirements.txt

RUN pip install --no-cache-dir --user -r /requirements.txt

COPY . /src
WORKDIR /src
ENTRYPOINT ["python", "-m", "dvc", "repro"]

This is how I run the container:

docker run --volume ${PWD}:/src --env=GOOGLE_APPLICATION_CREDENTIALS=<path_to_json> <image_name> ./dvc_configs/free/dvc.yaml --force

Everything works great when running this locally, but it fails when run on Gitlab CI/CD.

stages:
  - build_image

build_image:
  stage: build_image
  image: <internal_docker_repo_image>
  script:
    - echo "Building Docker image..."
    - mkdir ~/.docker
    - cat $GOOGLE_CREDENTIALS > ${CI_PROJECT_DIR}/key.json
    - docker build . -t <image_name>
    - docker run --volume ${PWD}:/src --env=GOOGLE_APPLICATION_CREDENTIALS=<path_to_json> <image_name> ./dvc_configs/free/dvc.yaml --force
  artifacts:
        paths:
          - "./data/*csv"
        expire_in: 1 week

This results in the following error: ERROR: you are not inside of a DVC repository (checked up to mount point '/src')

Just in case you don't know what DVC is, this is a tool used in machine learning for versioning your models, datasets, metrics, and, in addition, setting up your pipelines, which I use it for in my case.

Essentially, it requires two folders .dvc and .git in the directory from which dvc repro is executed.

In this particular case, I have no idea why it's not able to run this command given that the contents of the folders are exactly the same and both .dvc and .git exist.

Thanks in advance!

Answer 1

Your COPY . /src is problematic for the same reason as Hidden file .env not copied using Docker COPY. You probably need !.dvc in your .dockerignore.

Additionally, docker run --volume ${PWD}:/src will overwrite the container's /src so $PWD itself will need .git & .dvc etc. You don't seem to have cloned a repo before running these script commands.