I setup 2 k8s environments with minikube. One with the --container-runtime=docker flag and one with --container-runtime=containerd flag. Here are the differences I see.
When I set container-runtime=docker , these things happen
- there is a
dockerdservice that is running - The
dockerdservice spawnscontainerdas its own child - There are
/usr/bin/containerd-shim-runc-v2processes that run the actual containers, and the parent of each of thesecontainerd-shim-runc-v2is PID 1 on the system.
When I set container-runtime=containerd, these things happen
- there is no
dockerdservice, no ambiguities there. - there is a
containerdprocess, which is owned by PID 1. Again, no surprises there. - There are
containerd-shimprocesses that run actual containers, and the parent of each of thesecontainerd-shimprocesses iscontainerd
So here are my questions
- What are the differences between
containerd-shimandcontainerd-shim-runc-v2? They seem to take mostly similar flags etc. - Why in scenario 1 the shims are children of PID 1 whereas scenario 2 the shims are children of containerd ?
EDIT: Just thought of an edit. On a ubuntu 20 box, if I install docker, dockerd is a separate process whose parent is PID 1, containerd is a separate process whose parent is PID 1, and all containers are children of container-shim-runc-v2 whose PID is 1 ?!?! Why is containerd not a child of dockerd ? Where is this configured?
I have dug into this topic and came to the following conclusions and sources.
1. What are the differences between containerd-shim and containerd-shim-runc-v2? They seem to take mostly similar flags etc.
These are just different versions,
containerd-shim-runc-v2being the newest version of thecontainerd-shim. See the source code here.It looks like that docker still uses
containerd-shiminstead ofcontainerd-shim-runc-v2. The base functionality will still be the same function of the shim, being that the shim monitors runc containers to tell containerd when runc has finished a run time.If you are concerned about differences in the API, please reference the source code. But in functionality they are just different versions of the shim API.
2. Why in scenario 1 the shims are children of PID 1 whereas scenario 2 the shims are children of containerd?
Ultimately, they are both children of PID 1 where the shims are children of containerd which is a child of PID 1.
This blog post gives a good overview of runtimes on k8s and worker nodes. Particularly, the sections on Docker, containerd and shims will give more perspective on your questions.
Here is a more thorough resource into containerd, shims, and how they interact with linux.
And this resource dives into runc, containerd and their PIDs in linux.