SCD Type - 2 without use of inbuilt components

Nine Stars

SCD Type - 2 without use of inbuilt components

Hi,

Jus as the earlier post, i m trying scd -2 .

Source Table - dim_emp

emp_id,emp_name,emp_location

1,abc,london

2,def,boston

Target Table - tgt_dim_emp

emp_id,emp_name,emp_location,version

1,abc,london,1

2,def,boston,1

2,def,seattle,2

As per SCD - 2, the records have to be compared and the updated record must be inserted or updated.

Can anyone help me...how do i achieve in 3 formats - versioning,flagging,effective date

While trying for versioning... i have tried the following approach...

source table ---> tmap ---> target table(inserting all the values in the target table with version as 1).

onsubjobok

source table --->             tmap(In Tmap,how do i compare all the values of the table and how do insert the new record)----> target_table

target table(lookup) ---->

Pls do help.

An early repsonse would be appreciated.

Thanks,Ankit


Accepted Solutions
Eleven Stars

Re: SCD Type - 2 without use of inbuilt components

You could compare each field in Variable expression one by one . If all value not same , output two flows

1) update flow - lookup details with end date and is-active as N
2) insert flow- main flow details with version = version + 1 , effective date as current date time and is-active as Y
Regards
Abhishek KUMAR

All Replies
Eleven Stars

Re: SCD Type - 2 without use of inbuilt components

You could compare each field in Variable expression one by one . If all value not same , output two flows

1) update flow - lookup details with end date and is-active as N
2) insert flow- main flow details with version = version + 1 , effective date as current date time and is-active as Y
Regards
Abhishek KUMAR
Forteen Stars

Re: SCD Type - 2 without use of inbuilt components

@ankit7359 ,

please find the below links.

http://www.deepinopensource.com/scd-2-implementations-in-talend/

 

http://talenddev.blogspot.in/2016/03/apply-scd-without-using-scd-component.html

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now