Using kubernetes-dashboard with outh2-proxy and keycloak (and EKS) - unauthorized

Question

I am trying to use oauth-proxy to provide authentication on the kubernetes dashboard using keycloak in EKS.

I have managed to get to a point where oauth-proxy will forward the authorization header to the dashboard, however I am getting 'unauthorized' in the dashboard.

From what I can gather, the dashboard is expecting a field in the header with the id_token however the version of keycloak I am using does not seem to provide it... It is not clear to me if this is an issue with the way I have configured keycloak or an issue with something else.

Both oauth-proxy and kubernetes dashboard are deployed using helm with the following config:

kubernetes-dashboard:
  app:
    ingress:
      enabled: true 
      ingressClassName: nginx
      issuer:
        name: letsencrypt
        scope: cluster
      paths:
        web: /
        api: /api
      annotations: 
        external-dns.alpha.kubernetes.io/hostname: kubernetes-dashboard.example.com
        nginx.ingress.kubernetes.io/proxy-buffer-size: "64k"
        nginx.ingress.kubernetes.io/backend-protocol: HTTP
        nginx.ingress.kubernetes.io/auth-signin: 'https://kubernetes-dashboard.example.com/oauth2/start?rd=$escaped_request_uri'
        nginx.ingress.kubernetes.io/auth-url: 'https://kubernetes-dashboard.example.com/oauth2/auth'
        nginx.ingress.kubernetes.io/auth-response-headers: "Authorization"
      hosts:
        - kubernetes-dashboard.example.com

  nginx:
    enabled: false

  cert-manager:
    enabled: false
    installCRDs: false
  
  metrics-server:
    enabled: false

oauth2-proxy: 
  config:
    existingSecret: "kubernetes-dashboard-oidc-secret"
    configFile: | 
      provider="keycloak-oidc"
      provider_display_name="Keycloak"
      redirect_url="https://kubernetes-dashboard.example.com/oauth2/callback"

      email_domains = [ "*" ]
      oidc_issuer_url="https://keycloak.example.com/realms/myrealm"
      scope = "openid email groups"
      upstreams = [ "https://kubernetes-dashboard.example.com" ]

      cookie_secure = true

      set_authorization_header = true

  metrics:
    enabled: false

  ingress:
    enabled: true
    path: /oauth2
    className: nginx
    annotations: 
      nginx.ingress.kubernetes.io/proxy-buffer-size: "64k"
    hosts:
      - kubernetes-dashboard.example.com
    tls:
      - secretName: kubernetes-dashboard-tls
        hosts:
          - "kubernetes-dashboard.example.com"

  sessionStorage:
    type: redis
    redis:
      password: ""

  redis: 
    enabled: true 
    architecture: standalone

Versions

oauth2-proxy: Helm 6.16.1 (app 7.4.0)

kubernetes-dashboard: Helm 7.0.3 (app v3.0.0-alpha0)

Keycloak: 21.1.1

EKS Version: 1.26

EKS OIDC Config

Keycloak Config

I am using the same client in keycloak that I use for kubelogin using oidc (which does work).

EKS Logs*

I believe these are the errors for the dashboard - but sadly they are not clear...

jwt token

So when I visit https://kubernetes-dashboard.example.com/oauth2/callback I can see the authorization header but it does not contain an id_token...

When I get to the page - it logs in, however none of the resources are displayed... I am assuming that this is because the Authorization token is being passed and it does not contain the id_token, however nothing I have seemed to try appears to get it into the header... This would confirm why I am getting the errors in the API server...

Can anyone help?

Answer 1

so it turns out I had made an error in the OIDC configuration for EKS... As I am now using a later version of keycloak, I needed to make sure the issuer url was:

https://keycloak.example.com/realms/myrealm

and not

https://keycloak.example.com/auth/realms/myrealm

Frustratingly I had changed it everywhere else and not there... Everything now works as expected!

Thank you!!

Answer 2

You are correctly supplying a token with the required groups claim, so users can authenticate. But the K8S API server needs to know what K8S resources to grant access to, so users are unauthorized.

So you next need to deploy RBAC resources for each group. For example, apply this YAML to grant read access to all K8S resources to members of the admins group:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: dashboard-admins-role
rules:
- apiGroups: ['*']
  resources: ['*']
  verbs: ['get', 'list', 'watch']
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dashboard-admins-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: dashboard-admins-role
subjects:
- kind: Group
  name: admins

EDIT

Also, these were my oauth2-proxy settings in case it helps:

containers:
  - name: oauth2-proxy
    image: quay.io/oauth2-proxy/oauth2-proxy:v7.4.0
    args:
    - --http-address=http://0.0.0.0:4180
    - --upstream=http://kubernetes-dashboard:9090
    - --provider=oidc
    - --provider-ca-file=/oidc-ca-cert/oidcroot.ca.pem
    - --skip-provider-button=true
    - --oidc-issuer-url=https://login.mycluster.com/oauth/v2/oauth-anonymous
    - --client-id=kubernetes_client
    - --client-secret=Password1
    - --redirect-url=https://dashboard.mycluster.com/oauth2/callback
    - --code-challenge-method=S256
    - --scope=openid email kubernetes_read
    - --oidc-groups-claim=kubernetes_groups
    - --cookie-secret=pYLu3xdlvkYNw_e-TsvRI3iXFsXlnYrQ_yYGhChY_80=
    - --cookie-secure=true
    - --email-domain=*
    - --insecure-oidc-allow-unverified-email=true
    - --pass-authorization-header=true