One Star

Performace optimization in Talend

Hi Team,
Is there any performance optimization guide or tuning techniques for Talend. How to achieve any task to be completed in less time than required?. What is the best way for Tuning jobs to get better performance.
Any suggestions?

Regards,
Vijayvanamala
8 REPLIES
Seventeen Stars

Re: Performace optimization in Talend

There are a lot of possibilities to optimize jobs.
Starting with database components:
Activate batch mode in output components and activate fetch size in input components.
Switch on parallization...
Mostly the performance will be influenced be a clever job design. Never try to process all available data at once, better cut is in handy chunks and process them in iterations.
Avoid huge complex SQL joins and statements, try to join data in a tMap within the job.
Normally customers think about asking a consultant for help or a good idea is to book a training by Talend!
One Star

Re: Performace optimization in Talend

Thanks for your Suggestions Mr.jlolling.
IS there any document designed by Talend regarding Performance Optimization?.
Because i have a job where my source is OracleDB and Source is amazon S3 and there are millions of records to be moved form source to target (something in TB's) So, to improve the performance the above steps are sufficient or anything more is required to be done?.
Regards,
Vijayvanamala
Moderator

Re: Performace optimization in Talend

Hi,
Here is a online document about Talend Studio TalendHelpCenterSmiley Tongueerformance preferences, which works on the whole studio.
For a specific job, usually the performance is most based on DB connection, job design, component setting(Check Use Batch mode option).

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Performace optimization in Talend

HI Sabrina,
Thanks for sharing information about Performance tunning in Talend. Actually where I can find Component Settings(I am not using Talend Subcribed version) and how to activate batch mode in output components and also fetch size in input components.
With Warm Regards,
Viswanath.
Seventeen Stars

Re: Performace optimization in Talend

hi all,
have a look at hint option for Oracle Output 'to improve flow data'.
regards
laurent
Seventeen Stars

Re: Performace optimization in Talend

@vijayvanamala: To improve performance there is much more to be done as asking in a forum generic questions about "performance". You should think about a good design. After that you can implement your design in Talend and not vis versa.
Seventeen Stars

Re: Performace optimization in Talend

@jlolling
wise advice Smiley Happy
+1
regards
laurent
Four Stars

Re: Performace optimization in Talend

We have a case of a web service that for each received event triggers a lookup (tPostgresqlInput) to the database. Lookup is done with load at once option in the tMap component.
Our question is whether talend between event executions maintains its connection?
We are trying to minimize connection overhead that is done with each query. Any tips how to optimize this? Would a tConnection component make any difference?
Thank you.