Google Big Query view limitation on Python API

441 Views Asked by At

I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):

query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])
2

There are 2 best solutions below

0
On BEST ANSWER

I've just created a view with the Python Client Libraries using the following code:

from google.cloud import bigquery
client = bigquery.Client()
project = 'bigquery-samples'
source_dataset_id = 'reddit'
source_table_id = 'full'
shared_dataset_ref = client.dataset('my_dataset')
view_ref = shared_dataset_ref.table('my_shared_view')
view = bigquery.Table(view_ref)
sql_template = (
    'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
view.view_query = sql_template.format(
    project, source_dataset_id, source_table_id)
view = client.create_table(view)  # API request

print('Successfully created view at {}'.format(view.full_table_id))

And my view has 1359016 rows, verified by doing a:

SELECT COUNT(*) FROM `my_dataset.my_shared_view` 

Hope it helps.

0
On

Following is simple code which I tried and is working fine by me. Do try and let me know. Though I am using paid version.

from google.cloud import bigquery
client = bigquery.Client(project=dest_project)
query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'"%(TABLE+DATES[7])
job = client.query(query)
job.result()