SCD Type - 2 without use of inbuilt components

Nine Stars

SCD Type - 2 without use of inbuilt components

Hi,

Jus as the earlier post, i m trying scd -2 .

Source Table - dim_emp

emp_id,emp_name,emp_location

1,abc,london

2,def,boston

Target Table - tgt_dim_emp

emp_id,emp_name,emp_location,version

1,abc,london,1

2,def,boston,1

2,def,seattle,2

As per SCD - 2, the records have to be compared and the updated record must be inserted or updated.

Can anyone help me...how do i achieve in 3 formats - versioning,flagging,effective date

While trying for versioning... i have tried the following approach...

source table ---> tmap ---> target table(inserting all the values in the target table with version as 1).

onsubjobok

source table --->             tmap(In Tmap,how do i compare all the values of the table and how do insert the new record)----> target_table

target table(lookup) ---->

Pls do help.

An early repsonse would be appreciated.

Thanks,Ankit


Accepted Solutions
Eleven Stars

Re: SCD Type - 2 without use of inbuilt components

You could compare each field in Variable expression one by one . If all value not same , output two flows

1) update flow - lookup details with end date and is-active as N
2) insert flow- main flow details with version = version + 1 , effective date as current date time and is-active as Y
Regards
Abhishek KUMAR

View solution in original post


All Replies
Eleven Stars

Re: SCD Type - 2 without use of inbuilt components

You could compare each field in Variable expression one by one . If all value not same , output two flows

1) update flow - lookup details with end date and is-active as N
2) insert flow- main flow details with version = version + 1 , effective date as current date time and is-active as Y
Regards
Abhishek KUMAR

View solution in original post

Forteen Stars

Re: SCD Type - 2 without use of inbuilt components

@ankit7359 ,

please find the below links.

http://www.deepinopensource.com/scd-2-implementations-in-talend/

 

http://talenddev.blogspot.in/2016/03/apply-scd-without-using-scd-component.html

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog