I am having a use case of fetching data from a web service in an iterative fashion. I was wondering whether there is a way by which i may manually set the number of parallel threads to fetch data from the web service.
I did check out the tPartitioner component which helps me to set the number of threads up-to N-1, where N is the number of cores available in the machine.Hence, at max i am able to set 3 threads, since i have a quad core processor.
So , i am checking for options where i may set 20-40 parallel threads to fetch data from the web service.
Kindly let me know if there is any way to do this.
may i know how to get the thread ID of the parallel executions that are executed in the 'iterate' property?
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema