use different project for bigquery data than for the billing of queries

Question

I'm currently querying a project (call it data_project) in bigquery through the python SDK. For several reasons I want to restrict the access to this project to a full read-only and switch the queries itself to another project (call it quota_project) without having to specify the project in the query explicitly.

This means I don't want to do SELECT * FROM `project.dataset.table` but rather keep SELECT * FROM `dataset.table` while switching the project it is billed against.

Could be relevant here (but should not imho): quota_project has reservations.

To my understanding, this would be allowed by the option quota_project_id while initializing the credentials.

I tried the following two options which I thought make sense from following the documentation, neither of it worked out (both queries have been billed to the data_project):

import google.cloud.bigquery as bq
from google.oauth2.credentials import Credentials

creds = Credentials.from_authorized_user_file(credential_path, scopes)

# option 1
c = creds.with_quota_project(quota_project)
client = bq.client.Client(credentials=c)
client.query("SELECT 1").result().to_dataframe()

# option 2
client = bq.client.Client(
    credentials=creds, client_options={"quota_project_id": quota_project}
)
client.query("SELECT 1").result().to_dataframe()

Is anybody aware of how to achieve this? Or am I misunderstanding the intention of a "quota project"?

Answer 1

A quota project is used by client libraries, etc. for billing purposes. You can set the quota project using the CLI:

gcloud auth application-default set-quota-project my-quota-project

To set Application default credentials, you have to run the command:

gcloud auth application-default login

The existing application default credentials must have the serviceusage.services.use permission on the given project.

You can refer to this regarding to BigQuery quotas and limits and this stack link to know more about query costs in BigQuery.