I having a job like this
In my source oralce table 100 records are there and i loaded 100 records for first run and than 10 new records are added to the source.Now i run the job it only read the latest 10 records not all 100+10 records.How we can achieve this in talend.
There are several ways to have an incremental load.
If there is not too much data, you might consider using a tMap which inner joins existing data with all data and then have an output which catches the reject of inner join.
Please refer to talend CDC feature TalendHelpCenter:Change Data Capture (CDC): publish/subscribe which can capture only the changed source data and to move it from a source to a target system(s) in real time.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema