Google Cloud basic authentication: The caller does not have permission

Question

I am trying to authorize Dependabot for a private npm package repository stored in the Google Cloud Artifact registry with basic authentication. I created a Service Account and provided repository level read permissions by assigning the Artifact Registry Reader role.

The .github/dependabot.yml looks like this:

version: 2
registries:
  my-reg:
    type: npm-registry
    url: <my repo url>
    username: "_json_key_base64"
    password: "${{ secrets.KEY_BASE64}}"

updates:
  - package-ecosystem: "npm"
    registries:
      - my-reg
[...]

where secrets.KEY_BASE64 is the base64 encoded service account key stored as a Dependabot Secret in GitHub (following the official documentation).
I got some information about the authentication process from this StackOverflow Answer and this docker KEY_TYPE example.

I also tried this without the base64 encoding (username: "_json_key") but the error is still the same: {"error":"The caller does not have permission."}.

When I use a temporary token instead of the permanent secret key it works until the token expires:

[...]
  my-reg:
    type: npm-registry
    url: <my repo url>
    token: <my temp token>
[...]

When trying to access it with curl I noticed a similar behaviour. When a token is generated it works:

curl -H "Authorization: Bearer $(gcloud auth print-access-token)" <url>

but if I try using basic auth:

curl -u _json_key:<unencoded_secret_key> <url>

or with base64 encoding:

curl -u _json_key_base64:<encoded_secret_key> <url>

it does not work and results in the error: {"error":"The caller does not have permission."}. In both cases I removed all whitespace characters from the service account key.

Even with a .npmrc inside the root of the repository (where package.json is located), like mentioned here in the dependabot documentation, it still does not work:

@my-reg:registry=<url>

I tried using a temporary token (works, but unwanted since the token is temporary).
I tried using _json_key username with the unendcoded service account key (does not work).
I tried using _json_key_base64 username with the base64 encoded service account key (does not work).
I tried adding the above .npmrc file (does not work).

How can I provide a permanent solution for the Dependabot to authenticate to out private npm registry?