I am running statefulset: prometheus-stack/prometheus-ps-prometheus with 3 replicas (prometheus-ps-prometheus-0, prometheus-ps-prometheus-1, prometheus-ps-prometheus-2)
$ kubectl get statefulsets -n prometheus-stack prometheus-ps-prometheus
NAME
prometheus-ps-prometheus
$ kubectl get pods -n prometheus-stack -o wide
NAME STATUS IP
prometheus-ps-prometheus-0 Running 10.17.2.249
prometheus-ps-prometheus-1 Running 10.17.3.241
prometheus-ps-prometheus-2 Running 10.17.1.6
I have a headless service addresing the statefulset:
apiVersion: v1
kind: Service
metadata:
name: ptn
namespace: prometheus-stack
spec:
clusterIP: None
ports: [...]
selector:
app.kubernetes.io/name: prometheus
prometheus: ps-prometheus
I am trying to access the statefulset pods using the service DNS. When I address the service DNS I get correct IPs of the pods, but when I try to access the pods using the IPs I get error:
What am I doing wrong? How can I access the pods using the DNS?
$ dig ptn.prometheus-stack.svc.cluster.local
; <<>> DiG 9.18.13 <<>> ptn.prometheus-stack.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43817
;; flags: qr aa rd; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: ae0111659717eacd (echoed)
;; QUESTION SECTION:
;ptn.prometheus-stack.svc.cluster.local. IN A
;; ANSWER SECTION:
ptn.prometheus-stack.svc.cluster.local. 30 IN A 10.17.1.6
ptn.prometheus-stack.svc.cluster.local. 30 IN A 10.17.3.241
ptn.prometheus-stack.svc.cluster.local. 30 IN A 10.17.2.249
;; Query time: 4 msec
;; SERVER: 10.96.0.10#53(10.96.0.10) (UDP)
;; WHEN: Wed Jan 31 19:28:51 UTC 2024
;; MSG SIZE rcvd: 241
$ dig prometheus-ps-prometheus-0.ptn.prometheus-stack.svc.cluster.local
; <<>> DiG 9.18.13 <<>> prometheus-ps-prometheus-0.ptn.prometheus-stack.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 20283
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 7dfbe83a77ee64ef (echoed)
;; QUESTION SECTION:
;prometheus-ps-prometheus-0.ptn.prometheus-stack.svc.cluster.local. IN A
;; AUTHORITY SECTION:
cluster.local. 10 IN SOA ns.dns.cluster.local. hostmaster.cluster.local. 1706727875 7200 1800 86400 30
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10) (UDP)
;; WHEN: Wed Jan 31 19:29:31 UTC 2024
;; MSG SIZE rcvd: 199