Running Kubeadm from source build

Question

Attempting to deploy a k8 master node using kubeadm from a fork of the Kubernetes repository, branch release-1.19. What configuration is necessary ahead of running kubeadm init {opts...}

The kubeadm guide recommends install of kubeadm, kubectl and kubelet using apt. The guide states, following installation that "The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do."

From a local repository I'm compiling the Kubernetes binaries (kubeadm, kubectl and kubelet) using the 'make all' method. Then scp'ing them to the master node at /usr/local/bin with exec perms.

Executing kubeadm init fails since the kubelet is not running/configured. However, initialising the required kubelet.service from the kubelet binary seems to require the certs (ca.pem) and configs (kubelet.config.yaml) that I assumed kubeadm generates. So chicken-egg situation regarding kubeadm and the kubelet.

The question then is, what additional configurations does the apt installation complete for initialising the kubelet.service? Is there a minimal config & service template kubelet can be started with ahead of kubeadm init? Does kubeadm replace the certs used by the pre-initialised kubelet?

Any help/direction would be hugely appreciated. Online docs/threads for building from source are sparse

Answer 1

For anyone searching, found the solution to this:

1. Install dependencies through apt: apt-transport-https, conntrack, socat, ipset
2. Move kubelet, kubeadm, kubectl binaries to /usr/local/bin and give exec perms
3. Write systemd kubelet.service file to /etc/systemd/system

[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/home/
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/local/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target

4. Write kubelet config file to /etc/systemd/system/kubelet.service.d

# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
Environment="KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt"
Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=systemd"
ExecStart=
ExecStart=/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS

5. Build cni plugins https://github.com/containernetworking/plugins ie. For linux, build_linux.sh

6. Copy cni plugin binaries to /opt/cni

7. Start Kubelet systemctl daemon-reload systemctl enable kubelet --now systemctl start kubelet

8. Now kubeadm init can run

In short this initialised the kubelet.service systemd process prior to the kubeadm init; with some default/minimal configs. kubeadm init then modifies the process's configs on execution.