Use Docker Hub authenticated login for GitLab Runners to fix rate limiting issue

Question

am getting below pull rate limit issue error.

Running with gitlab-runner 14.0.1 (c1xsefsq)
   on gitlab-runners-xxxxxx   vjqSst4o
Preparing the "docker+machine" executor 01:42
 Using Docker executor with image node:14 ...
 Pulling docker image node:14 ...
 WARNING: Failed to pull image with policy "always": toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (manager.go:205:0s)
 ERROR: Preparation failed: failed to pull image "node:14" with specified policies [always]:
toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit (manager.go:205:0s)
Activity


To Fix pull rate limit issue with paid docker hub account authentication I have Modify the runner’s config.toml file as follows:
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#determine-your-docker_auth_config-data

[[runners]]
  environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"registry.example.com:5000\":{\"auth\":\"<NEW ACCESS TOKEN HERE FROM MY DOCKERHUB ACCOUNT>=\"}}}"]

how to fix this rate limit issue to use docker hub authentication for gitlab runner. I tried above configured DOCKER_AUTH_CONFIG in runner environment variable. in access token i used my docker hub access token. How to refer my token from aws param store in terragrunt code.

runners_environment_vars = ["DOCKER_AUTH_CONFIG={"auths":{"https://index.docker.io/v1/":{"auth":"MY TOKEN"}}}"]

Answer 1

I have used exactly the same way as you describe the DOCKER_AUTH_CONFIG and it works perfectly:

Running with gitlab-runner 15.8.2 (4d1ca121)
  on default-auto xa2gXnoZ, system ID: s_b8b3be06a37e
Preparing the "docker+machine" executor
Using Docker executor with image earthly/earthly:v0.6.30 ...
Starting service docker:dind ...
Authenticating with credentials from $DOCKER_AUTH_CONFIG
Pulling docker image docker:dind ...

Maybe Gitlab has fixed this issue in their docker+machine fork.

Nonetheless, after my build was failling due to the same issue while building my image so I solved it by following the described answer from this SO by adding a before_script which writes the content of the variable to the ~/.docker/config.json file and it works.