Performance Of job

One Star

Performance Of job

Hi Experts,
I am beginner for talend open studio and using 4.2.2 and i am getting severe performanc eproblem. Please see below scenario,
1. I am reading data more then 700K from the remote source which is oracle database using sql queries.
2. Using tMap I am loading fields from oracle table to thre different table in MySQL database.
This process is too slow it is processing almost 20-30 rows per second and its performance getting decreased with time. I have set cursor parameter for tOracleInput with 1500 raws also commiting 1500 rows on MySQL side.
Can you please help me how can i make it efficient and improve the performance.
Thanks and Regards,
Ketan

Re: Performance Of job

performance is the hardest metric to nail down because it involves so many details. A general strategy for debugging performance issues is to "isolate the bottleneck"
Start by reading data from oracle and sending it to a empty tJavaRow (blackhole). This will show your maximum read speed. If it is very slow, you know that there is some issue in the connection from the DB to you.
Then add your tMap to the flow-- if it slows down then you know something about your transform is the bottleneck.
if both of those are fast, and the job only slows down when you try to write to Mysql, it indicates that there is something amiss in the path from you to the Mysql database.
From what you describe, I suspect network issues-- but you need to do testing and isolate where the slowness is coming from.
One Star

Re: Performance Of job

Thank you John for your reply, I checked with tJavaRow component and this is faster and processing more then 1100 rows per second, as soon as I add tMap component this gone to ~800 rows per second even thaugh there is no transformation happening, its simply column matching like out of 10 select column three needs requires mapping. How can I optimize tMap component?
I am begineers in ETL tool and want to understand what would be the limit for talend data transformation, loading or reading per second? Is there any performance bench mark for talend and all its component is available anywhere?
Thanks and regards,
Ketan