I want to create a talend big data batch job where we need to compare 2 tables..tables have a primary key..while comparing if any changes are detected we need to update or insert in the second table..in short I want to do a real cdc .. please help me on this as I am a newbie to talend...
So far, talend CDC components are not available in bigdata batch job.
If you want to capture the changed data and only load these changed data into target table to achieve table sync, you can compare tables by using tMap.
Please refer to this online document about:TalendHelpCenter: Best Practice: Change Data Capture with Spark in Big Data.
Hope it will help.
Watch the recorded webinar!
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.