I am trying to run a scheduled DML query from our Firebase analytics dataset. (Backstory, we have a lot of extra events that are no longer needed and for the time being it is easier to drop them from BigQuery).
I thought I might be able to run the query such as the following:
DELETE FROM `my-project.analytics_1234567890.*`
WHERE _TABLE_SUFFIX
BETWEEN CONCAT('events_', format_date('%Y%m%d', date_sub(CURRENT_DATE, interval 7 day)))
AND CONCAT('events_', format_date('%Y%m%d', CURRENT_DATE))
AND event_name IN ('not a real event')
But, BigQuery complains with this error: DML over table my-project.analytics_1234567890.* is not supported
Is there a way to query dynamic table names in scheduled DML queries so I can drop specific rows from the last 7 partitions?
I wrote an article on how to resolve repetitive queries with scheduled invocations, combine simple queries to complex DML statements by using Cloud Workflows.
Automate the execution of BigQuery queries with Cloud Workflows
essentially it covers your use case:
Workflow Steps for your problem
workflow.yaml