Kubernetes FailedScheduling using nodeSelector

Question

I have set an On-Prem Kubernetes cluster using Rancher, with 3 Centos nodes and 1 Windows node. I wanted to set a Deployment that will never run over the Windows node, so I set in the Deployment spec.template.spec.nodeSelector: kubernetes.io/os: linux

It seems to run but the deployment gets stuck in Pending, with this error:

Warning FailedScheduling default-scheduler 0/4 nodes are available: 1 node(s) didn't match node selector, 3 node(s) had taint {cattle.io/os: linux}, that the pod didn't tolerate.

Any insights?

Answer 1

The scheduler is not able to schedule the pod on linux nodes because those nodes have got taints. So you need to add tolerations in the pod spec of the deployment.

  tolerations:
  - key: "cattle.io/os"
    operator: "Equal"
    value: "linux"
    effect: "NoSchedule"

Also add a specific taint to the windows nodes so that only specific pods with specific tolerations can only be scheduled onto the windows nodes

kubectl taint nodes windowsnode cattle.io/os=windows:NoSchedule