issue on avro file import in Google BigQuery

Question

I'm getting the following cryptic error message when trying to import an AVRO file created with fastavro into BigQuery:

Error while reading data, error message: The Apache Avro library failed to read data with the following error: Invalid branch index: 18446744073709551567, the leaves size is: 2 File:

I've searched all over the Internet, but I have no idea what this error actually means. Anyone have any ideas what could be the problem?

Answer 1

You can use the below code to load Avro data from Cloud Storage into a new BigQuery table.

from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set table_id to the ID of the table to create.
table_id = "project.dataset.table"

job_config = bigquery.LoadJobConfig(source_format=bigquery.SourceFormat.AVRO)
uri = "gs://cloud-samples-data/bigquery/us-states/us-states.avro"

load_job = client.load_table_from_uri(
    uri, table_id, job_config=job_config
)  # Make an API request.

load_job.result()  # Waits for the job to complete.

destination_table = client.get_table(table_id)
print("Loaded {} rows.".format(destination_table.num_rows))

You can refer to this document for more information.