What is the use case for ClusterPodMonitoring from in GCP prometheus engine

Question

ClusterPodMonitring defines how to scrape all pod monitors in cluster that have some label. In what namespace we sholud define this CR in kubernetes cluster. What is best pratice?

Somewhere I read it is used "This resource is a specialized tool intended for cluster-scoped metrics (such as kube-state-metrics)." Why in this case we cannot use PodMonitoring?

https://github.com/GoogleCloudPlatform/prometheus-engine/blob/v0.3.1/doc/

Answer 1

PodMonitoring is basically identical to ClusterPodMonitoring. They only differ by spec.scope in the CustomResourceDefinition like you can see in their GitHub repo. The latter is a cluster-wide definition, while PodMonitoring is namespaced. Depending on your needs you can freely choose which one to use - either you want to separate your scrape-jobs per namespace, or you don't.

A ClusterPodMonitoring resource can discover targets across all namespaces in a cluster, so best practice would be to spawn it in it's own namespace, e.g. named monitoring.

A PodMonitoring resource can only discover targets in it's own namespace.

Further on it's up to you which labels you want to be matched by (Cluster)PodMonitoring.spec.selector. It's always a good idea to maintain consistency throughout all clusters, whether using cluster-wide or namespaced monitoring.

---

tldr; A CRD in kubernetes can be defined in the scope Namespaced or Cluster (k8s docs), defining which resources they can access. Google did this with ClusterPodMonitoring.