How to detach or remove a data catalog tag from a BigQuery table using gcloud command

Question

Could anyone please share ETL tag template in GCP data catalog? I'd like to refresh a tag value with its ETL status every time a BigQuery table is updated. I'm trying to use gcloud commands to create a tag template. I need to remove the tag from the template using the gcloud command and add another tag to that template, so that I can maintain the ETL status through automation.

I am able to remove the tag through UI manually. I need corresponding gcloud command for the same.

Answer 1

The procedure is explained in the Data Catalog documentation:

1. Suppose that you have the following tag template created:

   gcloud data-catalog tag-templates create demo_template \
    --location=us-central1 \
    --display-name="Demo Tag Template" \
    --field=id=source,display-name="Source of data asset",type=string,required=TRUE \
    --field=id=num_rows,display-name="Number of rows in the data asset",type=double \
    --field=id=has_pii,display-name="Has PII",type=bool \
    --field=id=pii_type,display-name="PII type",type='enum(EMAIL_ADDRESS|US_SOCIAL_SECURITY_NUMBER|NONE)'
   

2. You need to lookup for Data Catalog entry created for your BigQuery table:

   ENTRY_NAME=$(gcloud data-catalog entries lookup '//bigquery.googleapis.com/projects/PROJECT_ID/datasets/DATASET/tables/TABLE' --format="value(name)")
   

3. Once you have entry name you can:

   • Create a tag if it wasn't created on the entry or if it was deleted earlier:

     cat > tag_file.json << EOF
     {
       "source": "BigQuery",
       "num_rows": 1000,
       "has_pii": true,
       "pii_type": "EMAIL_ADDRESS"
     }
     EOF
     
     gcloud data-catalog tags create --entry=${ENTRY_NAME} --tag-template=demo_template --tag-template-location=us-central1 --tag-file=tag_file.json
     

     The command returns (among others) tag name that can be used with update or delete.

   • Delete a tag from entry:

     gcloud data-catalog tags delete TAG_NAME
     

   • Update a tag, so that you don't have to delete it and recreate:

     gcloud data-catalog tags update --entry=${ENTRY_NAME} --tag-template=demo_template --tag-template-location=us-central1 --tag-file=tag_file.json
     

   • If you lost your tag name use list command:

     gcloud data-catalog tags list --entry=${ENTRY_NAME}