I have a talend job where I am loading data from a oracle source to a greenplum table. I have used tMap in between and the source row count is around 72 million. I have enabled parallel execution and the cursor from the source is also enabled. But still the data loading speed is not upto the mark and it's taking a lot of time to even load 10 million and then the connection is timing out. Could anyone please let me know how can I speed up the performance?
as I remember:
I am loading data from a oracle source to a greenplum table
You are copy data from one database to another database
and this component do this by 2 steps:
- export data to csv file
- run bulk insert command to import csv file
this combination work much faster in most of cases