[resolved] Create SCD without using specific SCD component

Hi,
Im almost sure that I saw this solved before in some blog/forum.. but currently Im not able to find it any more.
If I remember well it was using a tMap component.
I need to to make it in this way because my DW is on Redshift and it doesn't have any specific component for it.
Any help/link is apprenticed!
Thanks!
Oscar
1 ACCEPTED SOLUTION

Accepted Solutions
Seventeen Stars

Re: [resolved] Create SCD without using specific SCD component

hi all,
What about using SCD Talend component ?
https://help.talend.com/search/all?query=tMysqlSCD&content-lang=en
exist for other DB (Oracle,Postgres,etc)
hope it help
regards
laurent
PS : Forget !!! don't have read title well (without scd componenet) Smiley Wink ...
10 REPLIES
One Star

Re: [resolved] Create SCD without using specific SCD component

Can you be a bit more specific with your requirements.

Re: [resolved] Create SCD without using specific SCD component

Hi Janhess,
thanks for your answer! will try to explain, but Im looking for a generic solution that later I can apply to different scenarios.
I have a customer SCD dimension.
I need to implement the logic to insert and update depending of the new existent values in the staging table comparing it agains the dim table.
Some columns are Type 1
and others are Type 2
I just need to see some example that I can adapt later to my particular cases Smiley Happy
I think i can use a tMap component in order achieve it, but it will be great if I can find some kind of guide that I can use to achieve this.
Thanks!
Oscar
One Star

Re: [resolved] Create SCD without using specific SCD component

Is this the sort of thing you're looking for?
http://www.talendforge.org/forum/viewtopic.php?id=17304

Re: [resolved] Create SCD without using specific SCD component

Hi Janhess,
thanks for you link, at the end I based my solution in this video http://www.youtube.com/watch?v=dH567IFszu4
I think its closer to what Im looking for.
Now my solution works partially good, it works well when I have only SCD1 cases, and also when I have only cases for SCD2, but the tMap seems to not be able to finish when the SCD1 and SCD2 cases needs to be processed in the same run. Also not sure if this related with redshift that may not allow to perform updates from different components.
I can provide the transformation by email if necessary, but I dont know how to attach here in the forum.

Thanks!
Oscar

Re: [resolved] Create SCD without using specific SCD component

Hi,
I just added a printscreean where you can see how my solution looks like.
In order to avoid Redshift locks in the updates of Type 1 and Type 2, I splited the job in two parts.
(1) In the first one I do new Inserts and Updated Type1
(2) In the second part Im making Inserts and Updates Type 2
Cheers,
Oscar
Seventeen Stars

Re: [resolved] Create SCD without using specific SCD component

hi all,
What about using SCD Talend component ?
https://help.talend.com/search/all?query=tMysqlSCD&content-lang=en
exist for other DB (Oracle,Postgres,etc)
hope it help
regards
laurent
PS : Forget !!! don't have read title well (without scd componenet) Smiley Wink ...

Re: [resolved] Create SCD without using specific SCD component

Thanks Laurent for your suggestion, that will be great, but SCD component still don't exist for Redshift, which is what Im using now.
Cheers,
Oscar
Seventeen Stars

Re: [resolved] Create SCD without using specific SCD component

sorry so ... ask R&D to add it Smiley Wink !!
you can really ask it with talend tracker : https://jira.talendforge.org/secure/
One Star

Re: [resolved] Create SCD without using specific SCD component

i used the postgres scd for redshift and it seem to work but the performance is terrible! would you mind sharing your target load rows/s with your "custom" talend type 2 scd for talend? Would appreciate if you can provide some screenshots oscar!
One Star

Re: [resolved] Create SCD without using specific SCD component

Hi,
How to do SCD without SCD component??
Please share a screen shot or mapping
Thanks in advance
Regards,
Ramkumar
OJT-Walmart