Trying to use Terraform with GCP Workload Identity Federation returns `ACCESS_TOKEN_SCOPE_INSUFFICIENT`

458 Views Asked by At

I try to run Terraform to create a GKE cluster in GitLab CI/CD using the CI_JOB_JWT_V2 with Workload Identity Federation. Terraform returns ACCESS_TOKEN_SCOPE_INSUFFICIENT, and I have no idea why (Google did not help).

This is a copy of the GitLab CI output that shows that I managed to successfully authenticate with the desired service account.

$ gcloud iam workload-identity-pools create-cred-config ${GCP_WORKLOAD_IDENTITY_PROVIDER} --service-account="${GCP_SERVICE_ACCOUNT}" --output-file=.gcp_temp_cred.json --executable-command='cat ${CI_JOB_JWT_V2}'
Created credential configuration file [.gcp_temp_cred.json].
$ gcloud auth login --cred-file=`pwd`/.gcp_temp_cred.json --project=$GCP_PROJECT_ID
Authenticated with external account credentials for: [[email protected]].
Your current project is [vnagy-ec626bda].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID
$ gcloud auth list
                            Credentialed Accounts
ACTIVE  ACCOUNT
        ephemeral-runner@gitlab-ci-plan-free-3-35411a.iam.gserviceaccount.com
*       [email protected]
To set the active account, run:
    $ gcloud config set account `ACCOUNT`

Still, a few lines later, the same CI job returns

Terraform has been successfully initialized!
google_container_cluster.primary: Creating...
╷
│ Error: googleapi: Error 403: Request had insufficient authentication scopes.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "method": "google.container.v1.ClusterManager.CreateCluster",
│       "service": "container.googleapis.com"
│     },
│     "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT"
│   }
│ ]
│ 
│ More details:
│ Reason: insufficientPermissions, Message: Insufficient Permission
│ 
│ 
│   with google_container_cluster.primary,
│   on gke.tf line 1, in resource "google_container_cluster" "primary":
│    1: resource "google_container_cluster" "primary" {
│ 
╵

I checked the logs of the Workload Identity Pool and the Service account. Both logs are empty. This leaves me puzzled as authentication seems to work, so I'd expect some logs to show up.

Within GCP, the service account in question has the following permissions:

IAM for the service account

These permissions seem to be okay. I quickly created a cluster from the GCP web console using gcloud --impersonate-service-account=.... It worked.

0

There are 0 best solutions below