Lets assume we are laboring under the following requirements:
- We would like to run our code on gcp and wish to do so on kubernetes as opposed to on any of the managed solutions (app engine, cloud run...).
- Our services need to be exposed to the internet
- We would like to preserve client ips (the service should be able to read it)
- We would also like to deploy different versions of our services and split traffic between them based on weights
All of the above, except for traffic splitting, can be achieved by a traditional ingress resource which on gcp is implemented by a Global external HTTP(S) load balancer (classic). For this reason I was overjoyed when I noticed that gcp is implementing the new kubernetes gateway and route resources. As explained here they are in combination able to do everything that the ingress resource did and more, specifically, weighted traffic distribution.
Unfortunately however, once I began implementing this on a test cluster, I discovered that the gateway resource is implemented by 4 different classes, all of which are backed by the same two load balancers already backing the existing ingress resource. And only the two internal gateway classes, backed by gcp's internal HTTP(S) load balancer supports traffic splitting. See the following two images:
and
So, if we want to expose our services to the internet, we cannot traffic split, and if we wish to traffic split, we may only do so internally. I wasn't dismayed by this, it actually kind of makes sense. I mean ideally, the gke-l7-gxlb (backed by global external https load balancer (classic)) would support traffic splitting but I've encountered this sort of architecture in orgs before. You have an external load balancer, which does ssl termination and then sends traffic to internal load balancers which split traffic based on various rules. There are even images of such diagrams, using multiple load balancers on gcp's various tutorial pages. However, to finally return to the title, I cannot seem to be able to convince gcp's external load balancer to route traffic to the internal one. It seems to be very restricted in its backend definitions. And simply choosing an ip address (provided to us upon the creation of the gateway resource, i.e. the internal load balancer) does not appear to be an option.
Is this possible? Am I completely off base here? should I be solving this in a completely different way? Is there an easier way to achieve the above 4 checkboxes? I feel like an external load balancer sending traffic to an ip in my vpc network should be one of its most basic functions, but maybe there is a reason it's not allowing me to do this?