One Star tpk
One Star

Real Usage of tParallelize versus Enable Multi Thread execution in TOS

Hi All,
I want to know the real usage of tParallelize component. What is the actual purpose of tParallelize comopnent? By using this component will it increase the through put of the entire. Does it help in reducing the execution time of the job?
And if we do not use tParallelize component and just use Enable multi thread execution in TOS even this also executes the jobs in parallel, then what is the point in using tParallelize component.
Let's take a scenario, I have a master job which contain 50 tRunJob components offcourse where in each tRunJob contain the actual job which does the work. So in TOS in the master job i will leave all the tRunJob components unlinked and select Enable Multithread execution in job settings which will execute the jobs in parallel. The only potential risk i see is if the Ram is not sufficient enough we will face heap space errors and the job get failed. The above scenario is in TOS
Now, coming to enterprise edition in Master job we will use tParallelize component and connect all the tRunjobs with Parallelize links coming from the tParallelize component. So this will kick start all the subjobs at a time to run in parallel.
So the potential risk of heap space also comes here aslo if the job needs to execute big chunks of data. So what is the point of using tParallelize?
Please explain me the correct usage. I have read the scenario given in the user amnual for tParallelize but i am not satisfied with it. Please make me understand the concept clearly
With Regards,
Pavan Kumar Tummala
3 REPLIES
One Star tpk
One Star

Re: Real Usage of tParallelize versus Enable Multi Thread execution in TOS

Hi Team,
Is there any one who have good understanding of my above question? Please make me understand correctly the exact difference between tParallelize and Enable Multithread execution in TOS

Thank you in advance!
With Regards,
Pavan Kumar Tummala
One Star tpk
One Star

Re: Real Usage of tParallelize versus Enable Multi Thread execution in TOS

Hi All,

Is there any update on my doubt. Please clarify my doubt.
Thanks in advance!
With Regards,
Pavan Kumar Tummala
Community Manager

Re: Real Usage of tParallelize versus Enable Multi Thread execution in TOS

Hi tpk
Both tParallelize and 'enable parallel execution' allow you to run subjobs parallel, well, tParallelize also allows you to synchronizes the execution of a subjob with other sub-jobs within the main Job. For example,
tParallelize---parallelize--tRunJob_1
---parallelize--tRunJob_2
--synchronizes --tRunJob_3
in this example, tRunJob_1 and tRunJob_2 will run parallel, tRunJob_3 will run only when both tRunJob_1 and tRunJob_2 ends. So, tParallelize is the best component if you have a request that need some of subjobs to run parallel, and a subjob starts to run only when all other parallelize subjobs finishes. Also, tParallelize component makes your job design more flexible.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business