tParallelize - executing multiple jobs in parallel

Seven Stars

tParallelize - executing multiple jobs in parallel

We have designed a job which internally calls multple jobs to load files with certain layout to certain tables. In order to optimize the performance of our jobs we decided to use tParallelize component to run all these jobs in parallel. We made sure that "Multithread option" was checked for the same reason.
When job executed without any issues / errors - we decided to perform QA on populated tables in SQL Server. We found that column values have shifted or swapped for more than one table. There was a case where jobs skipped few rows from source file.
We then executed same job with different approach - run all jobs in sequence instead of in parallel. This time all jobs loaded data correctly for all files.
Screen shot for both the jobs are attached.
We believe this is bug is the component / TIS version we are using. We need to confirm this before putting any bug on jira. This issues is very critical for our project deliverables...
Community Manager

Re: tParallelize - executing multiple jobs in parallel

Hi
Which version are you using? tParallelize allows you to run different subjobs parallel, this component is only available in Talend enterprise subscription version, if you use tParallelize to run the subjobs parallel, don't need to check the "Multithread option", go to uncheck this option and run the job again, and see if you still get problems.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Seven Stars

Re: tParallelize - executing multiple jobs in parallel

Hi Shong,
I've tried all the possible combination including the one you've suggested. I'm using 5.2 Enterprise Edition. One more thing I observed while doing this for n number of times - if few column names of different files we are loading using different jobs are same; this issue occurs more frequently. Hope this gives some more clarity on this issue.
Thanks,
Nishad Joshi.
Community Manager

Re: tParallelize - executing multiple jobs in parallel

Hi Nishad
Thanks for your further explanation, but I will suggest you to report a ticket on our support portal, our colleagues from support team will help you further, we need to investigate if it was a bug or not, maybe a webex is needed to investigate the problem on live.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Seven Stars

Re: tParallelize - executing multiple jobs in parallel

Alright Shong,
Thank you very much for prompt reply - I'll submit a ticket on support portal for this. Will update this topic once I hear back from support team.
Thanks,
Nishad Joshi.
Seven Stars

Re: tParallelize - executing multiple jobs in parallel

ok, I submitted a ticket on support portal - I got reply that this is design issue, no further explanation. We tried to reproduce this scenario for different data set and it gave us similar results... so the issue is there with this component. Therefore went ahead and submitted a bug on jira -
https://jira.talendforge.org/browse/TDI-26239?page=com.atlassian.streams.streams-jira-plugin%3Aactiv...
If anyone else can work on this and re-produce this again that will be a great help.