Kubernetes Pod Affinity Topology Key for custom label

684 Views Asked by At

We're trying to deploy some Keda Scaled Jobs to our Kubernetes Cluster. We always want one node per job. I'm trying to configure a podAntiAffinity rule to enforce that a job doesn't get deployed to a node where another job (pod) with the given label is running. Per the documentation and samples, you need a topologyKey, which is

the key for the node label that the system uses to denote such a topology domain;

In our case we want to use an actual label that we specify, as opposed to, say, the hostname. How would we configure our rule in that case?

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: my-scaled-job-name
  namespace: my-namespace
  labels:
    # multiple jobs may have this label
    my-special-label: foo 
spec:
  ...
  jobTargetRef:
    ...
    template:
      spec:
        nodeSelector:
          "beta.kubernetes.io/os": linux
          agentpool: my-agent-pool
        affinity:
          # This anti-affinity rule makes sure a new pod won't start on a node where another one is already running
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            # Unclear whether this is the right topology key, or if I even need one?   
            - topologyKey: kubernetes.io/hostname 
              labelSelector:
                matchLabels:
                  - my-special-label: foo
        containers:
        - name: my-container
          ...
  # This is the trigger definition - watches the queue for incoming messages
  triggers:
    ...
0

There are 0 best solutions below