I have a daily partitioned table like so:
my_dataset.my_table_MMDDYYYY
When querying this table in the morning I get the following errors on multiple fields:
Cannot read optional field of type STRING as required STRING Field: id
Cannot read optional field of type TIMESTAMP_MICROS as required TIMESTAMP_MICROS Field: timestamp
When querying older partitions, I do not get the error. This table continues to get updated throughout the day.
Any idea what is going on here?
It appears that the "id" and "timestamp" fields were not required in some of the tables you are trying to select with a wildcard.
Those partitions must share the same exact schema, with optional fields.
I just encountered the same issue that started causing issues yesterday, although the schema changes happened in March 2022 and the query was successful until then.
The solution would be either to relax the REQUIRED constraints on the recent partitions or recreate the previous partitions with the same constraints.