Do you have tips to optimize a 10 million lines processing with a tSortRow before inserting it into the database?
I have good performance at the beginning (~ 6600rows / s), the more the number of treated lines increases, the more the performances decrease. Arrived at 600 000 lines, I have the error OutOfMemoryError: GC overhead limit exceeded (I could increase the memory of the JVM for the job, but I think it's not optimal)
Solved! Go to Solution.
Join us live for a sneak peek!
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.