Publishing container images to local Docker Desktop from Kaniko running in kubernetes

Question

I'm trying to noodle out how to make this basic workflow happen in Docker Desktop for Mac, with Kaniko running inside of a Kubernetes pod:

1. Pull a base image from the local Docker context
2. Build an image on top of that with other resources
3. Publish the new image back to Docker to the local context

with the goal of starting a new pod in Kubernetes that references the image that was just built, and having Kubernetes launch the pod with that new image.

We're already kind of doing this in a full-blown Kubernetes cluster where Kubelet is configured to pull from an external registry via an alias of cluster.local so instead of pulling from / publishing to Docker we're talking to that registry. The Dockerfile would resemble:

ARG REGISTRY_HOST=external.registry.fqdn
FROM ${REGISTRY_HOST}/path/to/my-base-image:1.0.0

COPY ...

RUN ...

...etc...

and we publish to external.registry.fqdn/path/to/my-new-image:2.0.0. But the image name spec for the pod would be cluster.local/path/to/my-new-image:2.0.0. So in Docker Desktop we'd need to be able to configure the build argument REGISTRY_HOST so that it points to the Docker Desktop context (I think).

This is primarily for being able to test the process locally, instead of having to push helm charts, etc. up to a cluster to test changes. Any suggestions on how one might go about doing something like this, perhaps with a local registry outside of Kubernetes and tweaks to kubelet to have it work like the full cluster configuration we're using would be greatly appreciated. If someone has managed to do something like this using colima and kubernetes, I will gladly take a look at that solution as well.

Answer 1

You can try these approaches to achieve the workflow locally on Docker Desktop:

1. You can setup Docker Desktop with kubernetes. You can enable kubernetes in Docker Desktop. This provides a local kubernetes environment for testing and development.

2. You can try to configure Kaniko to build and push images. Using Kaniko to build the docker image inside a kubernetes pod and push it to a local registry or Docker engine image cache.

3. You can try to update the dockerfile and kubernetes yaml file. In the dockerfile, use build arguments to specify the base image and destination registry. In the kubernetes yaml, reference the image that was built and pushed by Kaniko to the local registry or Docker Engine image cache.