Capture 1st rejected record and stop the job without processing anymore records

Seven Stars

Capture 1st rejected record and stop the job without processing anymore records

Hi
 I'm loading millions of records in to a oracle table using tOracleoutput component. while loading if for any reason a record gets rejected,
Is it possible to capture that record and stop the job.

I tired disabling the batch process and added a reject link to the toracleoutput and also added a "if trigger" link with condition                                          "((Integer)globalMap.get("tOracleOutput_1_NB_LINE_REJECTED"))> 0".I'm able to get the rejected records, but the job is not stopping until it process all the records.

is there anyway to stop the job without processing all the records?

 Untitled.jpg

 

Fifteen Stars TRF
Fifteen Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

Hi,

What if you connect your reject flow on the tDie component?


TRF
Seven Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

Tried that as well, the job is not stopping until it finished processing all the records.

Fifteen Stars TRF
Fifteen Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

Then you probably have to reduce the "Batch Size" parameter to 1.

Have a look to this post where I wrote something regarding the same issue for Salesforce component.

 


TRF
Seven Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

I have a doubt implementing your design, If I enable the "Use Batch SIze" in advanced setting, I can't have a reject link and its the same if I enable "Die on error" as well in tOracleoutput. So how to capture the rejected records and at the same time stop the job without processing anymore records?

Seven Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

Hi, Can anybody provide some insights on this pls?

Moderator

Re: Capture 1st rejected record and stop the job without processing anymore records

Hello,

Does your "RunIf" condition work well in your work flow?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

As i said earlier, even with the "If Trigger/Condition", the job is not stopping right away the moment it gets the first reject, its still processing all the records.

Moderator

Re: Capture 1st rejected record and stop the job without processing anymore records

Hello,


@borrisbeck wrote:

As i said earlier, even with the "If Trigger/Condition", the job is not stopping right away the moment it gets the first reject, its still processing all the records.


We suppose that your "RunIf" condition is not triggered successfully.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Capture 1st rejected record and stop the job without processing anymore records

Thanks for responding, this is the condition i'm using in the "If trigger", please let me know what is wrong 

"((Integer)globalMap.get("tOracleOutput_1_NB_LINE_REJECTED"))> 0"

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog