Reg: Restarting a job from the point of failure

One Star

Reg: Restarting a job from the point of failure

Hi All,
My source has 1 million rows and I should load these rows in to my target table. During the time of job execution 500,000 rows are getting comitted and my job fails with some data issue or unknown reason. When I restart the job it should fetch the records from 500001th row. Is any Talend component supports this feature or is it possible to do a work around method in Talend.
I am using TOS 3.2. My source is a CSV file.
I don't want to commit my rows after 1 million records. It should happen simultaneously (say for every 10000 records it should get commited)
Anxiuosly looking for some inputs
Thanks
Naju
Seven Stars

Re: Reg: Restarting a job from the point of failure

How often rows are committed is controlled by an Advanced Setting of the output component.
There is no way to get Talend to automatically restart from the previous point of failure. You would have to either:
1. Have a pre-job that counts the number of records in the output table, or
2. Have a post-job that OnSubjobError writes to a file the number of records output and have a pre-job that reads that file
so that you can use that count in the header parameter of the tFileInputDelimited component to restart from that point.
One Star anr
One Star

Re: Reg: Restarting a job from the point of failure

Hi
let us say I have 1 million records , and I want to commit upon change in column value (example sorted file with productID). can it be done?. suppose that I m using a tsybaseconnection for commit or rollback.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog