I am attempting to run a k3d cluster on a linux ubuntu machine using the following config:
{
"clusterToken": "123123123132",
"imageVolume": "k3d-tl-images",
"InitNode": null,
"kubeAPI": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "43077"
},
"host": "0.0.0.0",
"Port": "6443"
},
"kubeconfig": {
"switchCurrentContext": true,
"updateDefaultKubeconfig": true
},
"name": "tl",
"network": {
"id": "123123123123123",
"ipam": {
"ipPrefix": "172.19.0.0/16",
"ipsUsed": [
"172.19.0.1",
"172.19.0.3"
],
"Managed": false
},
"Members": [
{
"IP": "172.19.0.3",
"Name": "k3d-tl-registry"
}
],
"name": "k3d-tl"
},
"nodes": [
{
"agentOpts": {
},
"Cmd": [
"server"
],
"created": "2023-07-16T09:08:47.253631752Z",
"env": [
"all_proxy=",
"ALL_PROXY=",
"http_proxy=",
"HTTP_PROXY=",
"https_proxy=",
"HTTPS_PROXY=",
"no_proxy=",
"NO_PROXY=",
"K3S_TOKEN=123123123123",
"K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml"
],
"extraArgs": [
"--disable=traefik",
"--tls-san",
"0.0.0.0"
],
"ExtraHosts": null,
"GPURequest": "",
"image": "docker.io/rancher/k3s:v1.21.7-k3s1",
"IP": {
"IP": "",
"Static": false
},
"Memory": "",
"name": "k3d-tl-server-0",
"Networks": [
"k3d-tl"
],
"portMappings": {
"6443": [
{
"HostIp": "0.0.0.0",
"HostPort": "43077"
}
],
"80/tcp": [
{
"HostIp": "",
"HostPort": "4589"
}
]
},
"restart": true,
"role": "server",
"runtimeLabels": {
"app": "k3d",
"k3d.cluster": "tl",
"k3d.cluster.imageVolume": "k3d-tl-images",
"k3d.cluster.network": "k3d-tl",
"k3d.cluster.network.external": "true",
"k3d.cluster.network.id": "123123123123123",
"k3d.cluster.network.iprange": "172.19.0.0/16",
"k3d.cluster.token": "hjeriYqaurJuGhjUxVrg",
"k3d.cluster.url": "https://k3d-tl-server-0:6443",
"k3d.role": "server",
"k3d.server.api.host": "0.0.0.0",
"k3d.server.api.hostIP": "0.0.0.0",
"k3d.server.api.port": "43077",
"k3d.server.loadbalancer": "",
"k3d.version": "v5-dev"
},
"serverOpts": {
"kubeAPI": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "43077"
},
"host": "0.0.0.0",
"Port": "6443"
}
},
"State": {
"Running": true,
"Started": "2023-07-16T09:08:47.969985394Z",
"Status": ""
},
"volumes": [
"/var/lib/tl/standalone:/var/lib/tl/standalone",
"/home/tl/tl/data:/home/tl/tl/data",
"k3d-tl-images:/k3d/images"
]
}
],
"options": {
"disableLoadbalancer": true,
"globalEnv": [
"K3S_TOKEN=123123123"
],
"globalLabels": {
"app": "k3d",
"k3d.cluster": "tl",
"k3d.cluster.imageVolume": "k3d-tl-images",
"k3d.cluster.network": "k3d-tl",
"k3d.cluster.network.external": "true",
"k3d.cluster.network.id": "123123123123",
"k3d.cluster.network.iprange": "172.19.0.0/16",
"k3d.cluster.token": "123123123123",
"k3d.cluster.url": "https://k3d-tl-server-0:6443",
"k3d.server.loadbalancer": ""
},
"nodeHooks": [
{
"action": {
"Content": "123123123123123123",
"Description": "Write LocalRegistryHosting Configmap",
"Dest": "/tmp/localRegistryHostingCM.yaml",
"Mode": 420,
"Runtime": {
}
},
"stage": "preStart"
},
{
"action": {
"Content": "123123123123123123123123",
"Description": "Write Registry Configuration",
"Dest": "/etc/rancher/k3s/registries.yaml",
"Mode": 420,
"Runtime": {
}
},
"stage": "preStart"
}
],
"registries": {
"config": {
"Auths": null,
"Configs": null,
"Mirrors": {
"docker.elastic.co": {
"Endpoints": null,
"Rewrites": null
},
"docker.io": {
"Endpoints": null,
"Rewrites": null
},
"gcr.io": {
"Endpoints": null,
"Rewrites": null
},
"k8s.gcr.io": {
"Endpoints": null,
"Rewrites": null
},
"quay.io": {
"Endpoints": null,
"Rewrites": null
},
"us-central1-docker.pkg.dev": {
"Endpoints": null,
"Rewrites": null
}
}
},
"use": [
{
"ClusterRef": "",
"expose": {
"Binding": {
"HostIp": "0.0.0.0",
"HostPort": "5699"
},
"Port": "5000/tcp"
},
"host": "k3d-tl-registry",
"image": "sha256:123123123123123123123",
"options": {
"proxy": {
"remoteURL": ""
}
}
}
]
},
"waitForServer": true
},
"volumes": [
"k3d-tl-images"
]
}
And the command (golang):
k3dCluster.ClusterGet(ctx, runtimes.SelectedRuntime, &clusterConfig.Cluster)
Which results in the following error:
Failed Cluster Start: error during post-start cluster preparation: error overwriting contents of /etc/hosts: docker failed to attach to exec process in node 'k3d-tl-server-0': http: invalid Host header
I have tried manually setting permissions for the etc/hosts file, ensuring its valid, updating the http_proxy to be localhost and 127.0.0.1 and so on while always getting the same issue.
I see in the logs that the command docker is attempting to run when it fails with that error is:
Executing command '[sh -c cat /tmp/-etc-hosts-123123 > /etc/hosts]' in node 'k3d-tl-server-0'
Any assistance with understanding the cause and solution would be much appreciated
It could be this bug in docker.
To check, if it's the case, try doing
docker execto attach to a running container. Should you see that error, it's that bug.If it is the case: either downgrade, or wait until a fix is released
(by the operating system maintainers, or maintainers of an external repository, whatever applies).
I have this error with my current docker (24.0.4-ce built with go1.20.6) when using the "docker exec" command.