Number limit by PODS in the same node on GKE

1.4k Views Asked by At

I'm trying to lift many containers in a large instance of Google Cloud with Container Engine.

I detect that only 104 containers are raised in the same node, the rest are pending with the following error with a describe command: kubectl describe po nginx-2243550509-nd1z7

  FirstSeen     LastSeen        Count   From                    SubobjectPath   Type            Reason                  Message
    ---------     --------        -----   ----                    -------------   --------        ------                  -------
    5m            5m              1       {default-scheduler }                    Warning         FailedScheduling        no nodes available to schedule pods
    7m            20s             27      {default-scheduler }                    Warning         FailedScheduling        pod (nginx-2243550509-nd1z7) failed to fit in any node

Is this a limitation of GKE?

Is it impossible to lift more containers per instance?

Can you specify the internal IP range of containers on each node? (not the global IP range in the cluster)

I have the suspicion that only a /24 is allowed at each node.

That makes me think that the limitation could be 255 per node.

--

The test was done with the image container-vm (Debian 7) and GCI (Chromium OS).

The same limitation has also been observed with versions 1.5.1 and 1.4.7 of Kubernetes

--

UPDATE:

Seems a default limitation of the kubelet that is in each node, and I suppose that in GKE will be of 110 by default.

http://kubernetes.io/docs/admin/kubelet/

--max-pods int32     Number of Pods that can run on this Kubelet. (default 110)

Is it possible to modify this value in GKE in some way to use the entire available IP range?

Supposedly on /24 range.

1

There are 1 best solutions below

2
On

you can have a maximum of 252 nodes because of the ip range of /24 and you can have a total of 110 pods per node, this last thing is because Kubernetes assigns each node a range of IP addresses, a CIDR block, so that each Pod can have a unique IP address.