I am trying to create multiple worksheets in an excel workbook and the number of worksheets is not known at the start of the job. This is dependent on the values of a field from my Oracle input. I followed the post to design the same and this works wonders when there is very less data. Even with 18k records from my toracleinput, the iterations completed when run for whole night are just about 500.
I tried to modify the approach using toraclerow (Prepared statement setting) and tparserecordset and couldn't see anything better. I have attached both the approaches design.
The rest of the job completes in about 10 minutes and just the part of final excel creation is the problem now. How can I improve the performance. My toracleinput has a simple SELECT query that return 18k records.
Here exists a custom component tFileExcelSheetOutput written by talend community user and shared on talend exchange portal.
Could you please have a look at it to see if it helps?