I have a job that takes in data from sql server and puts them in mysql. I have used tmysqlconnection, tmysqlcommit and tmysqlrollback.
Similar to the above I have 15+ jobs. All these jobs must run in parallel
What problems might I face if the opening/closing of mysql connection happens 15+ times simultaneously?
Thanks in advance
Solved! Go to Solution.
No problems at all, unless you are restricted by the number of connections that your MySQL database allow. You must tweak your mysql database to allow the number of connections you have per job X the number of jobs you are running.
Look also at the performance of your DB when you have 15+ jobs trying to insert/update/delete data to your DB. Your DB can become the bottleneck if it cannot cope with the throughput thrown at it.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.