Is it possible to interact Cloud SQL or Cloud Spanner with Google Data Catalog

Question

I am able to discover Bigquery datasets,GCS files in Google Data Catalog but I could not find Cloud SQl or Cloud Spanner options in Cloud Data Catalog UI.

Is it possible to view Cloud SQL tables , Cloud Spanner tables data in Data Catalog? If yes please suggest steps or provide documents links.

Thanks.

Answer 1

Yes, It is Possible Details: Other than Native-Metadata types GCS, PUB/SUB and BigQuery are needs to be dealt via Catalog-APIs

Ref: https://cloud.google.com/data-catalog/docs/how-to/custom-entries

ie, Use One of 7 languages to programatically loop-through all the tables from custom data source (eg..BigTable) and create Tag Dynamically.

My Favorite Python & C#

Much appreciated if anyone else has better alternative approach

Answer 2

Unfortunately, there is no a native integration between Data Catalog, Cloud SQL and Cloud Spanner. Nevertheless, there is an issue tracker regarding this feature reported.

As you can see in the shared link, as a work around you can manually create a JDBC connector to Spanner and export the metadata to Data Catalog custom entries on a schedule. Something like the Mahendren's suggestion. Something similar you can perform with Cloud SQL

Answer 3

Yes, It is Possible using Data Catalog custom entries.

To view Cloud SQL tables, you can use the open source connectors for MySQL, SQL Server and PostgreSQL.

Also check the on-premise ingestion use cases from the official docs.