SCD3 implementation with more than two level of history

Highlighted
Five Stars

SCD3 implementation with more than two level of history

Hi everyone

I'm new to talend and new to the community.I want to capture the history of particular attributes something like we do SCD type3, but with more than a single level of history. please help me with your expertise how can I achieve it with talend.

I've data something like this

id, Region,State

1250 ,South,TN
1251 ,East,WB
1252 ,South,AP

 

and I want store history like below

id, Current_Region,previous_lvl1_Region,previous_lvl2_Region,previous_lvl3_Region,Current_State,previous_lvl1_State,previous_lvl2_State,previous_lvl3_State

1250 ,South,null,null,TN,null,null

 after first change  it must look like this (change are region = East )

1250 ,East,South,Null,TN,Null,Null

after second change it must look like this (change are region = North and state=HR)

1250 North,East,South,HR,TN,Null

 

 

 

 


Accepted Solutions
Highlighted
Nine Stars

Re: SCD3 implementation with more than two level of history

Please find the attached document for job flow and results. Regards,
Veeru Boppudi

View solution in original post


All Replies
Highlighted
Nine Stars

Re: SCD3 implementation with more than two level of history

Please find the attached document for job flow and results. Regards,
Veeru Boppudi

View solution in original post

Highlighted
Five Stars

Re: SCD3 implementation with more than two level of history

Thanks Sir, for such a quick reply and it's almost what I needed but you don't considered the when only state may be changed not the region and it have to store the history just for the state and region should be null as region is not changed so no value will be entered in previous_lvl1_region 

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog