GCP Identity Federation Authentication not working from docker on github actions

Question

I'm using the GCP python sdk from docker, this works locally by passing in the application_default_credentials.json file and using that to auth.

From github actions, using identity federation, I cannot get it to work. I'm getting a readtimeout error for some reason.

If I use a generated service account creds json file by passing it in as a github secret it works. It's only with identity federation that it doesn't work.

My workflow file.

  integration-tests:
    runs-on: uhg-runner
    permissions:
      contents: "read"
      id-token: "write"
      pull-requests: "write"
    env:
      USER: "nonprod"
    steps:
      - name: checkout repo code
        uses: actions/checkout@v3
      - name: "Authenticate to Google Cloud"
        id: "auth"
        uses: "google-github-actions/auth@v2"
        with:
          workload_identity_provider: "xxxx"
          service_account: "xxxx"
          token_format: "access_token"
          create_credentials_file: true
      - name: Set up Cloud SDK
        uses: "google-github-actions/setup-gcloud@v2"
      - name: 'Use gcloud CLI'
        run: |
          gcloud info
      - name: test
        run: docker compose run --rm api coverage run  --source=./src -m pytest tests/integration

docker-compose.yml has the following volume

volumes:
  - $GOOGLE_APPLICATION_CREDENTIALS:/tmp/keys/gcp.json

with the env variable

 - GOOGLE_APPLICATION_CREDENTIALS=/tmp/keys/gcp.json

I have confirmed GOOGLE_APPLICATION_CREDENTIALS env variable is set correctly on github actions and points to a json creds file formatted like this.

{"type":"external_account","audience":"//iam.googleapis.com/projects/xxxx/locations/global/workloadIdentityPools/gh-pool/providers/gh-provider",
"subject_token_type":"urn:ietf:params:oauth:token-type:jwt",
"token_url":"https://sts.googleapis.com/v1/token",
"credential_source":{"url":"https://pipelinesghubeus24.actions.githubusercontent.com/xxxx..../idtoken?api-version=2.0&audience=https%3A%2F%2Fiam.googleapis.com%2Fprojects%2F140501271167%2Flocations%2Fglobal%2FworkloadIdentityPools%2Fgh-pool%2Fproviders%2Fgh-provider","headers":
{"Authorization":"***"},"format":{"type":"json",
"subject_token_field_name":"value"}},
"service_account_impersonation_url":"https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/xxxxxx...us.iam.gserviceaccount.com:generateAccessToken"}

I'm getting this error in the integration test that is trying to run a query against bigquery.

self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f114160c7a0>
method = 'GET'
url = '/xxxx/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/A...gleapis.com%2Fprojects%2F140501271167%2Flocations%2Fglobal%2FworkloadIdentityPools%2Fgh-pool%2Fproviders%2Fgh-provider'
body = None
headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-aliv...xxxx'}
retries = Retry(total=2, connect=None, read=None, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=120, read=120, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None, preload_content = False
decode_content = False, response_kw = {}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/xxx/ojects%2F140501271167%2Flocations%2Fglobal%2FworkloadIdentityPools%2Fgh-pool%2Fproviders%2Fgh-provider', fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False
err = ReadTimeoutError("HTTPSConnectionPool(host='pipelinesghubeus24.actions.githubusercontent.com', port=443): Read timed out. (read timeout=120)")
clean_exit = False

Any idea why this is happening or how I can debug it further?

Answer 1

I think you are having error on connecting with the workload identity pool

err = ReadTimeoutError("HTTPSConnectionPool(host='pipelinesghubeus24.actions.githubusercontent.com', port=443): Read timed out. (read timeout=120)")
clean_exit = False

You may check the following links that can be helpful with your troubleshooting

• Full CI CD on Github Actions enabling Keyless Authentication and Workload Identity

• Integrate Cloud Run and workload identity federation

• Google GitHub Actions