Six Stars

How to improve the loading performance

a.PNG

 

Help me how to improve the loading performance. i have 1 lakhs record in a table and moving into destination table.

It is loading very slow. I am using Talend open studio 6.4.1

4 REPLIES
Twelve Stars

Re: How to improve the loading performance

performance in this case depends from target database

 

You can examine settings and configuration for check how them relevant for task

 

also You can test job with tMySQLBulkExec instead of tMySQLOutput

-----------
Six Stars

Re: How to improve the loading performance

I am doing insert/update option . it is not possible using tmysqloutputbulkexec.

 

So can you tel me how to speed up the loading process 

Nine Stars

Re: How to improve the loading performance

In/Update is your problem... only a redesign of your process by doing inserts only and rejected records to update...

Or... wifi / network speed limits your throughput.
Twelve Stars

Re: How to improve the loading performance

insert/update - always slow operations, and You can not expect any chances to improve it (yes, it possible tune database, but it will be not huge difference)

You can try redesign job - there are many variant depending from data:
- You can bulk import all into database (into separate table), and use
- or  update where exists + insert where not exists
- or delete where exists + insert all

P.S. it is bad practice to use local terms on international forum if You not expect answers in French or Chinese (I can also start use Russian terms ;-) ) ... but in any case 100 000 records - it is not too big for run inside database

P.P.S. of course staging table must have indexes

-----------