SCD Start and End Date

654 Views Asked by At

enter image description here

I am new to ssis and i am a little bit confused about the SCD_Start_Date and SCD_End_Date when we use the SCD dimension wizard. on many examples that i have searched online people always use the "system start time" or "system container " start time. but i am implementing a data warehouse from a scratch but our company have multiple data sources from two or three years ago. so when i upload our data source from these two years into our dw after running the package it will obviously show that, for example, a seller (seller Dimension) changed from department in the day i ran that package but this is wrong because i want to have a reference date that is in the data source (each row has a data_Ref) . isn't this the logic or am i missing something? how do you guys in BI upload old data source with a reference date?

1

There are 1 best solutions below

0
On

SCD Start and End dates are the dates that a version of the record became (or stopped being) the current version in your source system - they have nothing to do with when data was moved between environments (unless that movement was part of a transaction that changed the state of a record).

Going forward, it is relatively easy to create new SCD records and you insert a new version of a record (and update the previous current version of the record) every time there is a change in the source system.

Loading historic data is entirely dependent on your source system. If it only holds the current version of a record then you can only load that single version into your DWH. If the source system, contains a history of the changes made (or, for example, an audit trail of changes made) then you should be able to re-construct the history in a way that allows you to load multiple versions of a record into your DWH