Seven Stars

CDC on files

Hi Folks,

 

Anyone has idea how to acchieve the CDC between 2 files. I am looking for direct/indirect solutions without loading data to database.

 

As my database is Redshift and currently no CDC feature supported by talend for Redshift database.

 

Kindly post your thoughts

 

Thanks,
Sid
Mark as solution if this resolved your issue
4 REPLIES
Six Stars

Re: CDC on files

Hi sid,

Try doing a left outer join with source as primary and target as the ref link based on key cols. All the null values will be as insert. And others wil be an update. Nite ref should be a full file.

Thanks,
Ram
Six Stars

Re: CDC on files

I agree with brama! We have redshift as output and we do the same. Thanks.
Seven Stars

Re: CDC on files

Thank you guys for reply. I am aware of that approach.

What i am looking is is there any component/custom code if anyone developed for CDC comparisons between files.

Also i need to find the I,U,D,New records between both files.

Thanks,
Sid
Mark as solution if this resolved your issue
Six Stars

Re: CDC on files

I have not researched on any custom components on CDC as we were able to achieve what we were looking for (Inserts and updates) and deletes is something which we didn't needed as part of our requirements.