403 Error when accessing Google Secrets Manager from GKE

Question

I'm trying to use Google Secrets Manager to store database credentials for a Golang application running in GKE in Autopilot mode.

I've followed the steps recommended by Google to create the cluster, create the service account within GKE and associate it with an IAM account on Google with the correct permissions.

The steps I took for this are as follows:

Create the Cluster

gcloud config set project goproject
gcloud container clusters create-auto goproject-develop-gke --project goproject --region us-central1

Associate the service account with the correct roles:

gcloud iam service-accounts add-iam-policy-binding [email protected] \
    --member "serviceAccount:goproject.svc.id.goog[default/goproject-sa]" \
    --role "roles/iam.workloadIdentityUser"
gcloud projects add-iam-policy-binding goproject \
    --member "serviceAccount:[email protected]" \
    --role "roles/secretmanager.secretAccessor"

Create the service account in GKE

kubectl create serviceaccount goproject-google-sa

Annotate the service account with the IAM role

kubectl annotate serviceaccount goproject-google-sa \
    iam.gke.io/gcp-service-account=goproject-service-account@goproject.iam.gserviceaccount.com

Apply the following deployment script (image name redacted for privacy)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: goproject-service
spec:
  selector:
    matchLabels:
      app: goproject-service
  replicas: 1
  template:
    metadata:
      labels:
        app: goproject-service
    spec:
      containers:
      - name: goproject-service
        image: REDACTED
        ports:
        - containerPort: 5000
        env:
          - name: PROJECT_SERVICE_PORT
            value: "5000"
          - name: PROJECT_DB
            value: project_db
          - name: PROJECT_ID
            value: "REDACTED"
        resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
              ephemeral-storage: "1Gi"
            limits:
              memory: "1Gi"
              cpu: "500m"
              ephemeral-storage: "1Gi"
      serviceAccountName: goproject-google-sa
      nodeSelector:
        iam.gke.io/gke-metadata-server-enabled: "true"

However, when I apply the deployment and run it, I'm getting a 403 Error in the code:

rpc error: code = Unauthenticated desc = transport: per-RPC creds failed due to error: compute: Received 403 `Unable to generate access token; IAM returned 403 Forbidden: Permission 'iam.serviceAccounts.getAccessToken' denied on resource (or it may not exist).

The following code snippet shows how I'm trying to access the secrets using the official Google libraries:

// Get environment variables from Pod spec.
projectID := os.Getenv("PROJECT_ID")

// Create the Secrets Manager Client
client, err := secretmanager.NewClient(context.Background())
if err != nil {
    log.Println(err)
    return credentials
}
defer client.Close()

// Get the Secret
req := &secretmanagerpb.AccessSecretVersionRequest{
    Name: "projects/" + projectID + "/secrets/cockroach/versions/1",
}
res, err := client.AccessSecretVersion(context.Background(), req)
if err != nil {
    log.Println(err)
    return credentials
}

Have I missed a step somewhere? I've checked the IAM page and the permissions have been added.