Our system produces a lot of Delta files that we would like to query in Dremio. The files might be updated with new schema.
Is there any way for us to programmatically clear format of a delta table and then reformat it in Dremio?
Our system produces a lot of Delta files that we would like to query in Dremio. The files might be updated with new schema.
Is there any way for us to programmatically clear format of a delta table and then reformat it in Dremio?
I believe you can refresh the Delta Lake table metadata using this SQL:
See https://docs.dremio.com/software/data-formats/delta-lake/#refreshing-metadata