Five Stars rm
Five Stars

Talend JVM in runtime

We are using Talend Big data Enterprise edition.
In our jobs, volume of data subject to change. Sometime, we will process 1 million and other time 10 million. Job is getting failed due default JVM memory settings.
I want to change JVM memory settings in runtime, depending on the amount of data we process. Triggering the job through unix. How to handle this case while running through shell? Kindle guide me.
Thanks
5 REPLIES
Moderator

Re: Talend JVM in runtime

Hi,
So far, you can not change JVM setting when build a job in studio.
You can export your jobs as Standalone Jobs(.sh or .bat) and change JVM setting from commandline.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars rm
Five Stars

Re: Talend JVM in runtime

Thanks Sabrina.
Right now, our .sh has default settings as 
java -Xms256M -Xmx1024M
But this need to be changed in runtime, while running the job depending on the amount of data, we are processing.
Thanks
Seventeen Stars

Re: Talend JVM in runtime

You cannot change this setting while the job is running! It is not a lack of ability of Talend, it is simply the fact, these settings are initial settings for the JVM right at the start.
It is not a problem to allow much more memory with this parameters. The memory will only be allocated if it is needed.
Five Stars rm
Five Stars

Re: Talend JVM in runtime

Thanks jlolling.
If i run with default settings in .sh file, job fails due to out of memory issue for huge dataset.
java -Xms256M -Xmx1024M
You want us to allocate more memory(maximum anticipated depends on data in future)?. Any other way to handle this? Kindly help me.
Moderator

Re: Talend JVM in runtime

Hi,
Have you tried to allocate more memory to your job to deal with your out of memory issue before running?
You can open the Job which you want to allocate more memory to and in the Run view, open the Advanced Settings tab and select the Use specific JVM arguments box. Then allocate more memory to the active Job by double clicking the default JVM arguments and editing them.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.