multithreading for large xls file

One Star

multithreading for large xls file

Hi,
I have excel file with 10M records.
when i try to load it into DB with multithreading enabled and disabled it doesn't show much difference in excecution time.
then i found
https://help.talend.com/search/all?query=How+to+automatically+enable+parallelization+of+data+flows+f...
article.
I want to test if this works but it says i need to be subscribed to one of the Talend Platform solutions or Big Data solutions V5.3.1 or later.
how can i test that??
Moderator

Re: multithreading for large xls file

Hi,
 Parallelization of data flows feature is only available in Talend Platform solutions.
Feel free to send an email to sale@talend.com with your requirement. Our colleagues from sale team will see if it is possible to provide you with a free trial Talend Platform product for testing.

I have excel file with 10M + records.
when i try to load it into DB with multithreading enabled and disabled it doesn't show much difference in excecution time.


How did you set your t<DB>output component? Have you tried to change "Commit every" option to get better performance at execution?

Best regards
Sabrina

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: multithreading for large xls file

Hi,
I currently I am using Talend Open Studio for Data Integration. version : 5.6.1.XXX .
are you saying I need to purchase license In order to avail this feature???
How did you set your t<DB>output component? Have you tried to change "Commit every" option to get better performance at execution?

yes...I tried various combinations of commit every and batch execution .
we are more interested in Parallelization of data flows . I will contact talend sales team as you suggested.
Thanks,
Pankaj
Moderator

Re: multithreading for large xls file

Hi,
So far, talend provides community users with a free trial (30days) Talend Enterprise for Data Integration.
For your requirement(we are more interested in Parallelization of data flows),  you have to use Talend Platform solutions.
Feel free to send an email to sale@talend.com with your requirement. Our colleagues from sale team will see if it is possible to provide you with a free trial Talend Platform product for testing.

As we suggested before , please contact talend sales team to see if there is a free trial Talend Platform product for testing.
Don't hesitate to let us know if it is OK with you.
Best regards
Sabrina


Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: multithreading for large xls file

Hi,
I have sent mail to your sales team.now I am waiting for response.
Thanks,
Pankaj
One Star

Re: multithreading for large xls file

hi,
I installed Talend Enterprise for data integration V5.6.2 with license for 30days.
I still cant get set parallelization option.
am I missing something??? Is the version wrong ???
must subscribed to one of the Talend Platform solutions or Big Data solutions V5.3.1 or later.

what does this mean exactly ..
p.s. - no response from your sales team.
thanks,
Pankaj
Moderator

Re: multithreading for large xls file

Hi,
 
For your requirement (we are more interested in Parallelization of data flows),  you have to use Talend Platform solutions.
 
This feature is only available in Talend Platform Solutions not Talend Enterprise for Data Integration.
 
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Moderator

Re: multithreading for large xls file

Hi,
Do you get any response from talend  sales team?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: multithreading for large xls file

Hi,
No.I haven't got any response from talend sales team.
Thanks,
Pankaj

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch