Call Joblet inside tLoop component

Five Stars

Call Joblet inside tLoop component

Hi Dears,

 

Why i can't link an "iterator" from tLoop in my personalized Joblet?

I've a JobMain (my parent Job). Into my JobMain i have:

tLoop (it loops from 1 to 10) ---> iterator ---> Joblet (child job)

 

I can't link both, iterator and Joblet call.

 

There is any solution to do it?

 

 

Tks,

Clayton

Community Manager

Re: Call Joblet inside tLoop component

There is a trick to this. Use a "Trigger_Input" in the Joblet. Connect the "Trigger_Input" to whatever component you need to start the process using an OnComponentOK link. The connect your tLoop to a tJava (use it as a dummy component). Then link the tJava to the Joblet using an OnComponentOK link. This should work for you.

Four Stars

Re: Call Joblet inside tLoop component

This doesn't work for every scenario. Because the loop and joblet are not officially part of the same subjob the loop sometimes wants to execute before the rest of the sequence completes. That causes a failure in some cases. Seems there needs to be some additional support for joblets to take an iterate link.

Community Manager

Re: Call Joblet inside tLoop component

In what situations does this not work? I use Joblets like this all the time. However, another workaround which will work is to do the following. 

 

Use the INPUT component with the Joblet and create either a dummy schema of 1 column, or pass your data from your job to it. If you are linking from a tLoop, link the tLoop to a tFixedFlowInput with the same schema as the INPUT component. Once you have that done, it is just a case of working the Joblet to use your data from the INPUT OR (if a dummy column is used) link the INPUT to a tFlowToIterate and use the iterate link to initiate the main flow in your Joblet. Essentially doing this you have .....

 

tLoop------->tFixedFlowInput---row---->INPUT---row---->tFlowToIterate---iterate--->{the rest of the job}

If you iterate from a tLoop, if there are 10 loops and each iterates 10 times, you will have 100 iterations taking place in batches of 10.

Community Manager

Re: Call Joblet inside tLoop component

Actually, thinking about this some more, you must be mistaking the cause of your issue. The reason I say this is that SubJobs follow a strict order of processing. OnSubJobOK links will fire only once a SubJob is complete. OnComponentOK links will fire within the execution cycle of the previous subjob. If you have a SubJob (1) connected to another SubJob (2) by an OncomponentOK link, and then link that SubJob (2) to another Subjob (3) via an OnSubJobOk link, and to finish off have the first SubJob (1) linked via an OnSubJobOK to another SubJob (4), the processing will be in this order...

 

SubJob (1)

|------SubJob (2)

|                  |--- SubJob (3)

SubJob (4)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download