How can I retrieve dataplex data profile results to BigQuery?

497 Views Asked by At

I have been exploring google cloud dataplex data profiling feature here

I followed the steps given in creating the data profile scans and noticed the profile results are shown in the Profile section as per this documentation

If I want to retrieve those profile results to BigQuery, what options does gcp provides. I did check some of the apis, but did not notice any details to export profile scans to bigquery.

I am fairly new to using google cloud, therefore I may be missing something.

1

There are 1 best solutions below

1
On

Exporting dataplex data profile results to BigQuery is supported now.

Please check the public doc at Step Optional: Export the scan results to a BigQuery standard table for more details.

You can either bring your own table (which schema needs to be compatible with the required schema) or let the dataplex created the table for you if the table doesn't exist.

Please make sure the export BigQuery dataset exists and assign the dataplex service account corresponding permissions.