I have a talend job where I am loading data from a oracle source to a greenplum table. I have used tMap in between and the source row count is around 72 million. I have enabled parallel execution and the cursor from the source is also enabled. But still the data loading speed is not upto the mark and it's taking a lot of time to even load 10 million and then the connection is timing out. Could anyone please let me know how can I speed up the performance?
as I remember:
I am loading data from a oracle source to a greenplum table
You are copy data from one database to another database
and this component do this by 2 steps:
- export data to csv file
- run bulk insert command to import csv file
this combination work much faster in most of cases
Watch the recorded webinar!
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.
Practical steps to developing your data integration strategy.