Trying to execute multiple subjobs but one subjob is not fetching data.

Four Stars

Trying to execute multiple subjobs but one subjob is not fetching data.

Hi ,

 

I am trying to execute multiple subjobs in a parent job. 

Subjob1 --Hasoutput1 

|

|

Subjob2 -- Hashoutput2

|

|

Hashinput1----     tunite ---- logrow

Hashinput2----

 

All these jobs are executing sequentially.  When i am executing the subjob2 individually(not along with other jobs) it is executing fine. But when i am executing it along with otherjobs(as mentioned above).

Then subjob2 is not fetching any records(zero). See the attachment. 

 

Thanks

Manav Adwal

 

 

Eight Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi @manavadwal ,

 

Can you please let us know how you are passing data from Sub Job to parent job.

 

Thanks and Regards,

Subhadip

Highlighted
Four Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi Subhadip13,

Thanks for your quick reply.

 

I am executing the nested subjobs using the runjob component.

Subjobs are passing the data to parent using tbufferout component.

 

So the every runjob is getting populated by child subjob(independent).

 

See the attachment. 

If you want to know how the child jobs are getting populated. The answer is child jobs are either getting populated by individual job or child job is executing the subjob using trunjob.

And every child job is having the tbufferout component at the end to pass the data to parent job.

 

Thanks 

Manav Adwal

 

Tags (1)
Four Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi Subhadip13,

 

See the attachments.

Four Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi Subhadip13,

 

See the attachments.

Eight Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi @manavadwal ,

 

It is always better to keep the data in some file and use the file in parent job if you have large amount of data.

 

Both Hash and Buffer are memory consuming components hence your job performance will also get impacted.

 

Can you please put the data of your subjobs into files and use the files in parent job to do the necessary operations.

 

Thanks and Regards,

Subhadip

Four Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi Subhidip13,

 

Thanks for your reply.

 

But i want to do it without inserting the data into files. I know i can follow this path but i dont want to do that. i want to execute the job without generating the files.

One more thing i would like to share is if i swap the subjobs which means if i execute the second subjob first and first subjob second then second job will execute fine and first job will not load the data.

 

Which means it will start misbehaving. so every time whatever the second job is it will not load the data.

 

I just want to understand why this is happening and what is the solution of this issue?

 

Thanks

Manav Adwal

 

Four Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi Subhidip13,

 

Can you provide me the solution of this issue.

 

Or anyone who is aware of this issue can help me regarding this.

 

Regards.

Manav Adwal

 

 

Eight Stars

Re: Trying to execute multiple subjobs but one subjob is not fetching data.

Hi @manavadwal ,

 

My suggestion would be to use files instead of buffer and hash, which will boost your performance, enhance the debug operation and increase scalability and readability.

 

Thanks and Regards,

Subhadip

 

 

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog