Effective way to track source deleted records in incremental load in Azure Databricks

42 Views Asked by At

Scenario :

  1. The source is snowflake and it is in readonly mode.
  2. Implementing full load, saving maximum timestamp in databricks for incremental load.
  3. But in source side apart from record updates, records also being deleted and those records also need to be deleted from target(Databricks).
  4. Note: dont want to do full load on regular basis as it is not effective.

Any helpful tips with example ?

0

There are 0 best solutions below