Talend time optimization via bat file

Highlighted
Five Stars

Talend time optimization via bat file

I have a series of jobs which I call via a .bat file for scheduling. The bat file includes includes the jar files which i use to call the talend jobs, however the job is taking a lot of time to process the data (approx 3 hrs) to complete jobs. Any suggestions to make the jobs fast?

Highlighted
Community Manager

Re: Talend time optimization via bat file

Hi
Could you upload a screenshot of your job? We will see if it is needed to optimize your job design and give you comprehensive advice.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Sixteen Stars
Sixteen Stars

Re: Talend time optimization via bat file

The problem is not the way used to launch the jobs. The real question is what do they have to do?

TRF
Highlighted
Employee

Re: Talend time optimization via bat file

Hi,

 

    You might be having lot of data volume but the memory parameters given in the job might be minimal. Also you need to check whether you have optimized your lookups. It will be always better if you can reduce the result set at the beginning of the job itself. 

 

     Sharing the job design will help us to understand your job better and can give more precise thoughts.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog