[resolved] Talend Sub-Job is hanging quite long time in "Starting"

One Star

[resolved] Talend Sub-Job is hanging quite long time in "Starting"

Hi,
I hope you can help me further...
I have created a simple job (see attached screen shot below) using tFlowToIterate for looping (one sequence number after the other is loaded and compared to the last valid row of data).
As higher the sequence number gets, as lower the number of rows handled in one loop (first ~ 200.000 rows, last  ony one row).
While number of rows were high, the runtime of the job seemed to be good, but with low number of rows in the loop the processing itself takes milliseconds but the starting part takes several seconds.
Since I have to process arount 21.000 loops this time when the sub job is "Starting" is really causing major runtime issues.
Do you have any idea wht I can do to reduce the "Starting" time?
Thank you and best regards
Sven


Accepted Solutions
One Star

Re: [resolved] Talend Sub-Job is hanging quite long time in "Starting"

Hi,
thanks for reply!
I found the main reason. In tables used for testing one index was not created, the most important one for this job on the sequence number.
Creating the index it runs very quick.
Regarding Commit4, thanks for Hint. You are right, it does nothing.
Best regards
Sven

All Replies
Seventeen Stars

Re: [resolved] Talend Sub-Job is hanging quite long time in "Starting"

hi,
check all sub-job 'performance' one by one first to see if slow 'answer' is from your condition or sql request.
I think that your commit 4 is done nothing.
For a better process use differents jobs for each data managment & a father job that control workflow and connection.

regards
laurent
One Star

Re: [resolved] Talend Sub-Job is hanging quite long time in "Starting"

Hi,
thanks for reply!
I found the main reason. In tables used for testing one index was not created, the most important one for this job on the sequence number.
Creating the index it runs very quick.
Regarding Commit4, thanks for Hint. You are right, it does nothing.
Best regards
Sven
Seventeen Stars

Re: [resolved] Talend Sub-Job is hanging quite long time in "Starting"

you're welcome Smiley Wink

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog