I setup 2 k8s environments with minikube. One with the --container-runtime=docker
flag and one with --container-runtime=containerd
flag. Here are the differences I see.
When I set container-runtime=docker
, these things happen
- there is a
dockerd
service that is running - The
dockerd
service spawnscontainerd
as its own child - There are
/usr/bin/containerd-shim-runc-v2
processes that run the actual containers, and the parent of each of thesecontainerd-shim-runc-v2
is PID 1 on the system.
When I set container-runtime=containerd
, these things happen
- there is no
dockerd
service, no ambiguities there. - there is a
containerd
process, which is owned by PID 1. Again, no surprises there. - There are
containerd-shim
processes that run actual containers, and the parent of each of thesecontainerd-shim
processes iscontainerd
So here are my questions
- What are the differences between
containerd-shim
andcontainerd-shim-runc-v2
? They seem to take mostly similar flags etc. - Why in scenario 1 the shims are children of PID 1 whereas scenario 2 the shims are children of containerd ?
EDIT: Just thought of an edit. On a ubuntu 20 box, if I install docker, dockerd is a separate process whose parent is PID 1, containerd is a separate process whose parent is PID 1, and all containers are children of container-shim-runc-v2 whose PID is 1 ?!?! Why is containerd
not a child of dockerd
? Where is this configured?
I have dug into this topic and came to the following conclusions and sources.
1. What are the differences between containerd-shim and containerd-shim-runc-v2? They seem to take mostly similar flags etc.
These are just different versions,
containerd-shim-runc-v2
being the newest version of thecontainerd-shim
. See the source code here.It looks like that docker still uses
containerd-shim
instead ofcontainerd-shim-runc-v2
. The base functionality will still be the same function of the shim, being that the shim monitors runc containers to tell containerd when runc has finished a run time.If you are concerned about differences in the API, please reference the source code. But in functionality they are just different versions of the shim API.
2. Why in scenario 1 the shims are children of PID 1 whereas scenario 2 the shims are children of containerd?
Ultimately, they are both children of PID 1 where the shims are children of containerd which is a child of PID 1.
This blog post gives a good overview of runtimes on k8s and worker nodes. Particularly, the sections on Docker, containerd and shims will give more perspective on your questions.
Here is a more thorough resource into containerd, shims, and how they interact with linux.
And this resource dives into runc, containerd and their PIDs in linux.