Run jobs in parallel

Run jobs in parallel


How can I run 3 jobs in parallel in another job?

I have
Job 1 on subjob ok -> job 2 on subjob ok -> Job 3 on subjob ok -> Job 4 parallel on subjob ok -> Job 5
Job 4 parallel is contains
Job 6, job 7 and job 8.
Just inserted in there without any trigger/row. I tought they were going to start to run once job 4 parallel starts?
Apparently he runs first job 6 and only when 6 is finished he runs 7, ...?
Seventeen Stars

Re: Run jobs in parallel

Real parallel run is only possible in the Professional or Enterprise edition with the tParallel component.
If you are limited to the Open Source Edition you have to design your jobs 6,7,and 8 in a way they runs for them self and polling on any trigger (like a file) to start their work.
This way you can scale your parallel processing as wide as you want.

Re: Run jobs in parallel

With Community version , it seems you are looking for "Multithread option" in your job? If so, check out "Multithreadoption" in Extra of job review then subjobs should be executed by Multithread without any connecter between them. .
See my pic.
The tParallelize allows you to run different subjobs parallel, this component is only available in Talend enterprise subscription, well, tParallelize also allows you to synchronizes the execution of a subjob with other sub-jobs within the main Job. For example,
--synchronizes --tRunJob_3
in this example, tRunJob_1 and tRunJob_2 will run parallel, tRunJob_3 will run only when both tRunJob_1 and tRunJob_2 ends. So, tParallelize is the best component if you have a request that need some of subjobs to run parallel, and a subjob starts to run only when all other parallelize subjobs finishes. Also, tParallelize component makes your job design more flexible.
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables


Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables


Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema