How to scrape metrics from postgresql helm deployment with kube-prometheus-stack

742 Views Asked by At

I have seamlessly been using kube-prometheus-stack for monitoring.

after realizing there is metrics: section in values.yaml file, I wanted to enable metrics for both existing deployments;

  1. postgresql (https://github.com/bitnami/charts/tree/master/bitnami/postgresql)
  2. cassandra (https://github.com/bitnami/charts/tree/master/bitnami/cassandra)

for cassandra deployment

altering the value of key enabled: from false to true was sufficient. After upgrading helm release with new values, a sidecar container is created. I confirmed that I do see the metrics displayed at /metrics. I see targets are listed and being scraped in Prometheus.

metrics:
  enabled: true

but for postgresql deployment

doing the same did not work; setting

metrics:
  enabled: true

resulted in creation of a sidecar container, metrics are displayable at /metrics. But it is not listed nor being scraped in Prometheus.

so my question is: why is this same setting giving the desired result for cassandra deployment but not for postgresql? am I missing something and what else do I need to check? Also, I don't need to enable serviceMonitor for these deployments(?) because prometheus can scrape pods based on annotations, right? Any help is appreciated.

annotation config

additionalScrapeConfigs in values.yaml file of kube-prometheus-stack is edited to work with prometheus.io/* annotations. (ref: Monitor custom kubernetes pod metrics using Prometheus)

prometheus:
  prometheusSpec:
    additionalScrapeConfigs:
      - job_name: 'kubernetes-pods'
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
        - source_labels: [ __meta_kubernetes_pod_annotation_prometheus_io_scrape ]
          action: keep
          regex: true
        - source_labels: [ __meta_kubernetes_pod_annotation_prometheus_io_path ]
          action: replace
          target_label: __metrics_path__
          regex: (.+)
        - source_labels: [ __address__, __meta_kubernetes_pod_annotation_prometheus_io_port ]
          action: replace
          regex: ([^:]+)(?::\d+)?;(\d+)
          replacement: $1:$2
          target_label: __address__
        - action: labelmap
          regex: __meta_kubernetes_pod_label_(.+)
        - source_labels: [ __meta_kubernetes_namespace ]
          action: replace
          target_label: kubernetes_namespace
        - source_labels: [ __meta_kubernetes_pod_name ]
          action: replace
          target_label: kubernetes_pod_name

versions

kubectl version: 
  v1.22.6
chart versions:
  kube-prometheus-stack-35.0.3
  postgresql-11.1.28
  cassandra-9.1.9
0

There are 0 best solutions below