Why can't I reach a pod in another namespace using the format http://<ip_with_dashes>.<namespace>.pod.cluster.local?

Question

I created a namespace and a pod in it like this:

# Create the namespace
kubectl create namespace one

# Create the pod (NOTE - Without a service)
kubectl run rest -n one --image nginx --labels app=rest --expose --port 80

I then created a second namespace and pod (that I'll use to reach the first one).

# Create the namespace
kubectl create namespace two

# Create the pod (NOTE - Without a service)
kubectl run call-rest -n two --image alpine -- sleep 3600

Then I grab the ip address of the first pod:

kubectl get pod rest -n one -o wide

# This gives me
# NAME   READY   STATUS    RESTARTS   AGE   IP              NODE               NOMINATED NODE   READINESS GATES
# rest   1/1     Running   0          10m   192.168.0.142   k3d-dev-server-0   <none>           <none>

I use that ip address to call it from the other pod.

# This uses the format that K8s creates for DNS
kubectl exec -it call-rest -n two -- wget -qO- --timeout=2 http://192-168-0-142.one.pod.cluster.local

The above just times out

wget: download timed out
command terminated with exit code 1

To verify, I checked the domain for the cluster and I have it correct:

# Run this inside the pod
kubectl exec -it call-rest -n two -- cat /etc/resolv.conf

# And it prints out:
#
# search two.svc.cluster.local svc.cluster.local cluster.local
# nameserver 10.43.0.10
# options ndots:5

The pods can reach each other if they're in the same namespace (but then I don't need to use the ip address).

Any idea how I can make them reach each other?

I'm using:

• k3d version v4.4.1
• k3s version v1.20.5-k3s1 (default)
• Calico

Answer 1

Turns out this is because I'm using Calico. With the default Flannel used in k3d, the above does work. However, in Calico it creates cluster-wide IP addresses for pods, not IPs within each namespace. So the following works:

kubectl exec -it call-rest -n two -- wget -qO- --timeout=2 http://192.168.0.142