Pin Kubernetes pods/deployments/replica sets/daemon sets to run on specific cpu only

Question

I need to restrict an app/deployment to run on specific cpus only (say 0-3 or just 1 or 2 etc.) I found out about CPU Manager and tried implement it with static policy but not able to achieve what I intend to.

I tried the following so far:

1. Enabled cpu manager static policy on kubelet and verified that it is enabled
2. Reserved the cpu with --reserved-cpus=0-3 option in the kubelet
3. Ran a sample nginx deployment with limits equal to requests and cpu of integer value i.e. QoS of guaranteed is ensured and able to validate the cpu affinity with taskset -c -p $(pidof nginx)

So, this makes my nginx app to be restricted to run on all cpus other than reserved cpus (0-3), i.e. if my machine has 32 cpus, the app can run on any of the 4-31 cpus. And so can any other apps/deployments that will run. As I understand, the reserved cpus 0-3 will be reserved for system daemons, OS daemons etc.

My questions-

1. Using the Kubernetes CPU Manager features, is it possible to pin certain cpu to an app/pod (in this case, my nginx app) to run on a specific cpu only (say 2 or 3 or 4-5)? If yes, how?
2. If point number 1 is possible, can we perform the pinning at container level too i.e. say Pod A has two containers Container B and Container D. Is it possible to pin cpu 0-3 to Container B and cpu 4 to Container B?
3. If none of this is possible using Kubernetes CPU Manager, what are the alternatives that are available at this point of time, if any?

Answer 1

After reading the documentation[1][2][3], and then doing some testing, the answer seems to be "no", at least with CPU Manager.

Despite this unclear snippet[1]:

This policy manages a shared pool of CPUs that initially contains all CPUs in the node. The amount of exclusively allocatable CPUs is equal to the total number of CPUs in the node minus any CPU reservations by the kubelet --kube-reserved or --system-reserved options. From 1.17, the CPU reservation list can be specified explicitly by kubelet --reserved-cpus option. The explicit CPU list specified by --reserved-cpus takes precedence over the CPU reservation specified by --kube-reserved and --system-reserved. CPUs reserved by these options are taken, in integer quantity, from the initial shared pool in ascending order by physical core ID. This shared pool is the set of CPUs on which any containers in BestEffort and Burstable pods run. Containers in Guaranteed pods with fractional CPU requests also run on CPUs in the shared pool. Only containers that are both part of a Guaranteed pod and have integer CPU requests are assigned exclusive CPUs.

What this actually means, as determined by testing, is that either reserved-cpus or kube-reserved and system-reserved can be set to dedicate CPUs that are not available for any pod to use. Setting kube-reserved and system-reserved will each be rounded up to the nearest whole CPU and have the corresponding number of CPUs removed from the set of all CPUs available to pods, starting with the lowest indexed CPUs when ordered by physical slot index. Or you can use reserved-cpus, which will then ignore kube-reserved and system-reserved, and you can specify physical slot indexes of CPUs to remove and reserve from what's available to pods. In either case, the remaining CPUs are the only ones available for pods to run on.

Separately, and in almost no way related to the prior paragraph it's part of, CPUs are allocated from the set of CPUs available to pods such that only Guaranteed QoS pods with whole number CPU requests will get whole CPUs exclusively dedicated to them. Basically it's just clarifying that pods meeting the requirements for the Guaranteed QoS category and that also want an even number of whole CPUs, can guarantee they won't be time-sliced across multiple CPUs throughout their lifetime, or have to compete with other pods being time sliced onto the CPU they are using. But in all other cases, pods may/will/can be time-sliced across multiple CPUs.

[1] https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/
[2] https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/
[3] https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/

Answer 2

As I understand your question, you want to set up your dedicated number of CPU for each app/pod. As I've searched.

I am only able to find some documentation that might help. The other one is a Github topic I think this is a workaround to your problem.

This is a disclaimer, based from what I've read, searched and understand there is no direct solution for this issue, only workarounds. I am still searching further for this.