What is difference between Minikube and built-in Kubernetes to Colima?

Question

I am exploring running Kubernetes cluster on developer environment (personal Mac M2 laptop). There are several ways to do that with different set of technologies (Minikube, Docker, Colima, Podman, Kind, etc).

For some reason I chose Colima as container runtime for Docker. Official Colima documentation says Colima has built-in Kubernetes, also it says following:

Yes, you can create a Kubernetes cluster with minikube (with Docker driver), Kind or K3d instead of enabling Kubernetes in Colima. Those are better options if you need multiple clusters, or do not need Docker and Kubernetes to share the same images and runtime. Colima with Docker runtime is fully compatible with Minikube (with Docker driver), Kind and K3d.

Therefor there are at least two options achieve my goal:

• use built-in Kubernetes to Colima via colima start --kubernetes
• run Minikube, using Colima under the hood.

I managed to configure and run successfully both ones. I am a bit confused and want to know pros/cons both setups. I noticed only different container set, that's all. Net surfing didn't help a lot.

What is difference between Minikube and built-in Kubernetes to Colima?

Answer 1

In the specific setup you're asking, they've made slightly different decisions around the specific way they choose to bootstrap a Kubernetes cluster.

Poking at this myself, Colima appears to have gone with using K3S as it's means of spinning up a single node cluster and serve the cluster API vs. Minikube's Kubernetes In Docker approach, AKA Kind.

Take the following output from a fresh Minikube install:

NAMESPACE     NAME                               READY   STATUS    RESTARTS       AGE
kube-system   coredns-787d4945fb-mlb7b           0/1     Running   2 (57s ago)    4m32s
kube-system   etcd-minikube                      1/1     Running   0              4m47s
kube-system   kindnet-dhrg4                      1/1     Running   0              4m33s
kube-system   kube-apiserver-minikube            1/1     Running   0              4m48s
kube-system   kube-controller-manager-minikube   1/1     Running   0              4m47s
kube-system   kube-proxy-2btkt                   1/1     Running   0              4m33s
kube-system   kube-scheduler-minikube            1/1     Running   0              4m48s
kube-system   storage-provisioner                1/1     Running   1 (4m2s ago)   4m46s

vs. Colima:

NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   coredns-6799fbcd5-s8wl4                   1/1     Running   0          12m
kube-system   local-path-provisioner-84db5d44d9-dlgtb   1/1     Running   0          12m
kube-system   metrics-server-67c658944b-bvpqj           1/1     Running   0          12m

in Minikube and Kind case, we see pods for the Kubernetes API server, scheduler, and controller manager along with ETCD among others whereas with Colima these pods are missing. The reason for this, paraphrasing the K3S docs, is that K3S encapsulates these into a single binary and makes use of a different storage provider than ETCD to serve the cluster API. Container networking is also different as well with Minikube by default using kindnet vs. K3S choice of flannel.

Now, what does that practically mean for you? Short answer, it depends. If you're looking to validate your app manifests are correct, both probably will be fine. Both get you started quickly enough. If you need to install certain things like am Ingress controller or a service mesh product, you may find certain choices from K3S or Kind that make it easier or harder. Only thing really to do is try both and see which gets you to where you want to be faster.