I try to run Terraform to create a GKE cluster in GitLab CI/CD using the CI_JOB_JWT_V2
with Workload Identity Federation. Terraform returns ACCESS_TOKEN_SCOPE_INSUFFICIENT
, and I have no idea why (Google did not help).
This is a copy of the GitLab CI output that shows that I managed to successfully authenticate with the desired service account.
$ gcloud iam workload-identity-pools create-cred-config ${GCP_WORKLOAD_IDENTITY_PROVIDER} --service-account="${GCP_SERVICE_ACCOUNT}" --output-file=.gcp_temp_cred.json --executable-command='cat ${CI_JOB_JWT_V2}'
Created credential configuration file [.gcp_temp_cred.json].
$ gcloud auth login --cred-file=`pwd`/.gcp_temp_cred.json --project=$GCP_PROJECT_ID
Authenticated with external account credentials for: [[email protected]].
Your current project is [vnagy-ec626bda]. You can change this setting by running:
$ gcloud config set project PROJECT_ID
$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
ephemeral-runner@gitlab-ci-plan-free-3-35411a.iam.gserviceaccount.com
* [email protected]
To set the active account, run:
$ gcloud config set account `ACCOUNT`
Still, a few lines later, the same CI job returns
Terraform has been successfully initialized!
google_container_cluster.primary: Creating...
╷
│ Error: googleapi: Error 403: Request had insufficient authentication scopes.
│ Details:
│ [
│ {
│ "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│ "domain": "googleapis.com",
│ "metadata": {
│ "method": "google.container.v1.ClusterManager.CreateCluster",
│ "service": "container.googleapis.com"
│ },
│ "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT"
│ }
│ ]
│
│ More details:
│ Reason: insufficientPermissions, Message: Insufficient Permission
│
│
│ with google_container_cluster.primary,
│ on gke.tf line 1, in resource "google_container_cluster" "primary":
│ 1: resource "google_container_cluster" "primary" {
│
╵
I checked the logs of the Workload Identity Pool and the Service account. Both logs are empty. This leaves me puzzled as authentication seems to work, so I'd expect some logs to show up.
Within GCP, the service account in question has the following permissions:
These permissions seem to be okay. I quickly created a cluster from the GCP web console using gcloud --impersonate-service-account=...
. It worked.