tMemoryMonitor

Five Stars

tMemoryMonitor

I've added this component to the Talend Exchange which I hope is a useful utility for monitoring memory of a running Job and, hopefully, helping with good memory management - avoiding Out of Memory Exceptions and also not wasting it.
All feedback welcome.
http://www.talendforge.org/exchange/?eid=1202&product=tos&action=view&nav=1,1,1
Community Manager

Re: tMemoryMonitor

Hi tal00000
It is a great component that helps us to monitor the memory usage of a running job. It will be better if the component can output the memory information to a log file.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: tMemoryMonitor

Hi tal00000
It is a great component that helps us to monitor the memory usage of a running job. It will be better if the component can output the memory information to a log file.
Shong

Thanks for the feedback.
That shouldn't be too difficult...
I'm also amending to show peak usage; although this can be already derived from the history.
Of course, it can only show the peak that is recorded at the time of sampling; but still useful.
Seventeen Stars

Re: tMemoryMonitor

Hi, the problem with memory leaks is the error is mostly not related to the component causing the problem.
I use a standalone damon which connects via JMX to the jobs and gets the memory state without the need of add something to the job.
Not applicable

Re: tMemoryMonitor

This is a really useful component.  Excuse my ignorance but what is the output telling me?
tMemoryMonitor_1: using 2347MB/3217MB (max=3641MB/64%) (processors=2)
<Name of the component>: <current memory use>/<?> (<max memory use allowed>) (<number of processors>)
The value after the "/" what is that?
Community Manager

Re: tMemoryMonitor

Hi clarkec21
Read the description of this component on Talend Exchange:
Monitor the memory usage of your running Job, with tMemoryMonitor
Information on the used, total and maximum (see -Xmx) memory of the Java runtime, are written to standard output or, if a warning threshold is reached, to standard error.
Options include ‘Enabled’, ‘Interval’ (milliseconds) and ‘Warning Threshold Percent %’.
Used in conjunction with your own diagnostic messages, this component will help you to identify where memory is being used within your Job an help you to avoid Out of Memory (OOM) Exceptions.
As well as ensuring that your Job has enough memory, you’ll also want to make sure that you do not waste system memory. The Talend Job defaults are -Xms256MB and -Xmx1024MB. As well as some of your Jobs requiring more that 1024MB, some will require less that 256MB. This utility will allow you to observe the peak memory usage and to tune accordingly.
For more information on Java Runtime memory reporting, read http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html.
Used memory is totalMemory - freeMemory. Percentages are based on used memory vs. maxMemory.

In your case, 2347MB is used memory, and 3217MB is total memory.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: tMemoryMonitor

Hi tal00000,
It's undoubtedly a very useful component to monitor the memory usage of a job at run time. But I need to know one thing that how one can fetch the information of this component in a file, it would be great if you can help me out. 
Thanks again!
Best Regards
Ananya
Six Stars

Re: tMemoryMonitor

Can some please explain me on what basis tMemoryMonitor calculates memory. In below example we see used, total and maximum memory has some value, so i wanted to understand on what basis these values are calculated.  
tMemoryMonitor_1: using 2347MB/3217MB (max=3641MB/64%) (processors=2)