[resolved] Log file on jobserver way too big

One Star

[resolved] Log file on jobserver way too big

Hi there,
Some of you might be using the technique where you log some messages using tWarn and/or tDie components along with tLogCatcher which is followed by your preferred output to a file or specific database.
This method works fine until you start running your jobs on a talend jobserver, then it starts to be interesting.
Each execution produces a folder with two files, stdOutErr_*.log and resuming_*.log , that is what is happening in my case.
Now the issue is the resuming_*.log could get quite big, depends on number of messages you send to the tWarn or tDie components as all of them go strait into the log file and I cannot find a way how to turn it off or at least filter only messages with Error or Fatal level set.
Please see my example attached.
The job executed on a jobserver produces these two logs:
stdOutErr_20140415095008_7fKhR.log
----------------------------------------------------------------------------------------
### Job STARTED at 2014/04/15 09:50:08 (jobId=20140415_095004_wxIll, jobExecutionId=20140415095008_7fKhR) ###
2014-04-15 09:50:09.398|This is to test TRACE
2014-04-15 09:50:09.407|2014-04-15 09:50:09.398|This is to test TRACE
### Job ENDED SUCCESSFULLY at 2014/04/15 09:50:09 (jobId=20140415_095004_wxIll, jobExecutionId=20140415095008_7fKhR) ###
----------------------------------------------------------------------------------------
resuming_20140415095008_7fKhR.log
----------------------------------------------------------------------------------------
"eventDate","pid","root_pid","father_pid","type","partName","parentPart","project","jobName","jobContext","jobVersion","threadId","logPriority","errorCode","message","stackTrace"
"2014-04-15 09:50:09.242","20140415095008_7fKhR","20140415095008_7fKhR","20140415095008_7fKhR","JOB_STARTED","JOB:test_tWarnAndCatch","","XXXXDEV","test_tWarnAndCatch","Default","0.1","","","","","","{""context_parameters"": {}}"
"2014-04-15 09:50:09.385","20140415095008_7fKhR","20140415095008_7fKhR","20140415095008_7fKhR","NODE","NODE:Implicit_Context_Context","","XXXXDEV","test_tWarnAndCatch","Default","0.1","","","","","","{""context_parameters"": {}}"
"2014-04-15 09:50:09.402","20140415095008_7fKhR","20140415095008_7fKhR","20140415095008_7fKhR","USER_DEF_LOG","NODE:tWarn_1","","XXXXDEV","test_tWarnAndCatch","Default","0.1","","TRACE","","2014-04-15 09:50:09.398|This is to test TRACE","",""
"2014-04-15 09:50:09.408","20140415095008_7fKhR","20140415095008_7fKhR","20140415095008_7fKhR","JOB_ENDED","JOB:test_tWarnAndCatch","","XXXXDEV","test_tWarnAndCatch","Default","0.1","","","0","","",""
----------------------------------------------------------------------------------------
Is there a way I could remove the line:
09:50:09.402","20140415095008_7fKhR","20140415095008_7fKhR","20140415095008_7fKhR","USER_DEF_LOG","NODE:tWarn_1","","XXXXDEV","test_tWarnAndCatch","Default","0.1","","TRACE","","2014-04-15 09:50:09.398|This is to test TRACE","",""
from the resuming log?
1) it says it is USER_DEF_LOG and level TRACE, but in real jobs this should be turned off and turned on when required.
2) it is already in our custom log already which has got the option On or Off implemented.
Thanks for any advice
Lubo

Accepted Solutions
One Star

Re: [resolved] Log file on jobserver way too big

For now the only work around I have is to limit the use of tWarn and tDie components - this will limit the size of the log.
Another major issue is that the implicit context load dumps all the loaded values into the log and if it contains sensitive and classified information, I definitely don't want to store them on this file.
So another work around and step as you said would be to delete the file and get rid off it, but again why should I do it and think of it as a developer?
Ideally I would like to see an option on the Jpbserver where I could turn off the resuming log file completely or have some sort of control what information it would log, it is enterpise after all!
Lubo

All Replies
Four Stars

Re: [resolved] Log file on jobserver way too big

Hi Lubod,
I am not sure how your messages are written in the file, but we can restrict the flow by using tFilter component and filtering out whatever we don't want for the screenshot you have given. This is what I do to restrict number of log rows which are inserted to database.
Thanks
Vaibhav
One Star

Re: [resolved] Log file on jobserver way too big

Yes that is what I normally do with the tMap component, you can see on my overview picture.
But it does not filter out messages from the resuming_20140415095008_7fKhR.log file, I do not have any control of what is written to it.
You can see the timestamp 09:50:09.402 is suggesting the resuming_*.log files are written before the tLogCather sends info to the tMap.
The tMap has got timestamp 09:50:09.407 - few milliseconds after.
Four Stars

Re: [resolved] Log file on jobserver way too big

I think you are working on Enterprise. This is generated by talend engine. As you said, we don't have any control on it.
I would recommend another approach...
- Create a new subjob for file handling
- read that resu*.log file,
- filter relevant contents,
- write into the another file and
- delete old file...
Do this periodically or may be at the end of job.
Vaibhav
One Star

Re: [resolved] Log file on jobserver way too big

For now the only work around I have is to limit the use of tWarn and tDie components - this will limit the size of the log.
Another major issue is that the implicit context load dumps all the loaded values into the log and if it contains sensitive and classified information, I definitely don't want to store them on this file.
So another work around and step as you said would be to delete the file and get rid off it, but again why should I do it and think of it as a developer?
Ideally I would like to see an option on the Jpbserver where I could turn off the resuming log file completely or have some sort of control what information it would log, it is enterpise after all!
Lubo
Four Stars

Re: [resolved] Log file on jobserver way too big

Have you checked log4j parameters in TAC?
One Star

Re: [resolved] Log file on jobserver way too big

No, I am not admin on the TAC and I do not have access to the servers.
You see, when I test any job in my local I do not see this issues because the UI does not produce these logs. It is only when the job is deployed and admins start complaining about resources, and they have no clue how to switch it off.
Four Stars

Re: [resolved] Log file on jobserver way too big

One Star

Re: [resolved] Log file on jobserver way too big

will do, thanks
One Star

Re: [resolved] Log file on jobserver way too big

Unfortunately the link did not help, but we found a solution somewhere else... http://www.talendforge.org/forum/viewtopic.php?id=16325
The link will give someone an idea how do disable recovery on the jobserver so the resuming log files are not produced.