oauth2-proxy not working with AWS Network Load Balancer on kubernetes(KOPS)

Question

Initially, I deployed Oauth2 single sign-on proxy(using GCP APIs and services) on my Kubernetes cluster(using KOPS) with an ingress-nginx-controller provisioned with a classic load balancer. The Oauth2 proxy works fine using the following annotations on the ingress resources:

nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth

Controller pod log shows 200 status and resources load fine on the web browser:

172.26.2.112 - - [11/Apr/2022:09:03:13 +0000] "GET /assets/images/resources/sa.svg HTTP/2.0" 200 535 "https://argocd.infra.example.com/applications/oauth2-proxy?resource=" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" 52 0.015 [argocd-argocd-server-443] [] 100.105.130.90:8080 535 0.016 200 8891ecb604a8db0edda63e184535793b
172.26.2.112 - - [11/Apr/2022:09:03:13 +0000] "GET /assets/images/resources/ing.svg HTTP/2.0" 200 369 "https://argocd.infra.example.com/applications/oauth2-proxy?resource=" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" 53 0.007 [argocd-argocd-server-443] [] 100.105.130.90:8080 369 0.008 200 4214b05d5b2cbadae0ae04b0f3393b21

But based on my team's requirement, I had to change my Ingress-Nginx-Controller provisioning AWS Network Load balancer instead of the default Classic Load Balancer and I did that now. I could expose all my apps service through NLB provisioned by KOPs.

But the issue now I am facing is the Oauth2 proxy or its annotations on ingress resources stopped working. Following are the logs with HTTP code 499 when using Oauth2 proxy with NLB controller.

100.115.172.192 - - [11/Apr/2022:09:00:00 +0000] "GET /oauth2/auth HTTP/1.1" 499 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" 2729 49.817 [argocd-argocd-server-443] [] - - - - 6efc0c5ade6cdfefd2e275f366bdcfd5
100.115.172.192 - - [11/Apr/2022:09:00:00 +0000] "GET /oauth2/auth HTTP/1.1" 499 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" 2729 49.809 [argocd-argocd-server-443] [] - - - - 6efc0c5ade6cdfefd2e275f366bdcfd5
172.26.2.112 - - [11/Apr/2022:09:00:00 +0000] "GET /oauth2/auth HTTP/1.1" 499 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.75 Safari/537.36" 2729 49.797 [argocd-argocd-server-443] [] - - - - 6efc0c5ade6cdfefd2e275f366bdcfd5

Is it because the controller load balancer changed from Classic(Layer 7) to Network Load Balancer(Layer 4) and the Oauth2 proxy simply cannot work or understand the requesting header this way?.

I could not find a workaround yet to have Oauth2 proxy working with NLB.

Answer 1

you might need this:

access_by_lua_block {
    ngx.var.pass_access_scheme = "https"
    ngx.var.pass_port = "443"
}