Is there any way to represent POD CPU usage in terms of CPU cores using prometheus metrics

Question

I just want to represent the CPU metrics of a POD as below

I am able to represent the CPU requests and limits in terms of CPU cores which are directly available through prometheus scrape metrics.

But in prometheus I don't see any direct metric to get the CPU cores used by a POD so, Can someone give me a work around or a way to represent the CPU usage of a POD in the terms of CPU cores.

Thanks in advance

Answer 1

The query you are looking for is this one:

sum(rate(container_cpu_usage_seconds_total{container_name!="POD"}[1m])) by (pod_name)

Here the explanation (from within to out, as Prometheus calculates this query):

• container_cpu_usage_seconds_total which gives you how long the CPU has been occupied. 1s = a single Core for a whole second
• {container_name!="POD"} ignores the meta cGroups.
• rate(....[1m]) gives you the value changes in the defined interval, here 1 minute
• sum(....) by (pod_name) add up all values which contain the same pod names, which will be the case if we have multiple containers in the same pod

For further information about Prometheus in Kubernetes you can read this blog here: https://blog.freshtracks.io/a-deep-dive-into-kubernetes-metrics-part-3-container-resource-metrics-361c5ee46e66

Answer 2

The container_name and pod_name labels have been renamed to container and pod starting from Kuberntes 1.16 - see this issue for details. So for Kubernetes 1.16+ the following PromQL query must be used for calculating per-pod CPU usage in CPU cores:

sum(rate(container_cpu_usage_seconds_total{container!~"POD|"}[5m])) by (pod)