Daily Rolling Logs for mdm-job.log

Five Stars

Daily Rolling Logs for mdm-job.log

Can someone point me to the location of, and correct configuration of the mdm-job.log? It appears the config is in {install}/mdm/conf/log4j-jobox.properties but I can't seem to get it setup to use the DailyRollingFileAppender.

Any advice appreciated.

 

Original:

log4j.rootCategory=INFO,CONSOLE,FILE

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%p [%c] - %m%n

log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE.File=${mdm.root}/logs/mdm-job.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.ConversionPattern=%d %p [%c] - %m%n

 

Modified:

log4j.rootCategory=INFO,CONSOLE,FILE

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%p [%c] - %m%n

##log4j.appender.FILE=org.apache.log4j.RollingFileAppender
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.FILE.File=${mdm.root}/logs/mdm-job.log
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
##log4j.appender.FILE.layout.ConversionPattern=%d %p [%c] - %m%n
log4j.appender.FILE.layout.ConversionPattern=%d{'.'yyyy-MM-dd} %p [%c] - %m%n

Moderator

Re: Daily Rolling Logs for mdm-job.log

Hello,]

If you are trying to capture MDM Log in Version 6.x, please check this list:

MDM Server: web application (/talendmdm) deployed on embedded application server JBOSS or Tomcat.

MDM Server logs are located at <MDM_ROOT>/logs where MDM_ROOT is typically the following:

Windows: MDM_ROOT can be C:\Talend\6.5\mdm

Linux: MDM_ROOT can be /opt/Talend/6.5/mdm

All MDM log files are cycled with the date attribute, e.g mdm.log.01/21/2019

How to download MDM Logs

*** Version 6.x ***

MDM Configuration File:

<MDM_HOME>/conf/mdm.conf

------------------------------------------

Datasources configuration file:

<MDM_HOME>/conf/datasources.xml

 

------------------------------------------

TDSC connection configuration file:

<MDM_HOME>/conf/tdsc-database.properties

------------------------------------------

MDM authentication configuration file:

<MDM_HOME>/conf/jaas.conf

------------------------------------------

MDM logs configuration file:

<MDM_HOME>/conf/log4j.xml

------------------------------------------

MDM jobs logs configuration file:

<MDM_HOME>/conf/log4j-jobox.properties

------------------------------------------

MDM application server configuration file:

<MDM_HOME>/apache-tomcat/conf/server.xml

------------------------------------------

MDM logs file (configurable):

<MDM_HOME>/mdm/logs/mdm.log

------------------------------------------

MDM jobs logs file (configurable):

<MDM_HOME>/mdm/logs/mdm-job.log

------------------------------------------

App Server logs:

<MDM_HOME>/logs/catalina.out

------------------------------------------

Let us know if it is OK with you.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Daily Rolling Logs for mdm-job.log

Thanks Sabrina. Confirming that the configuration for mdm-jobs.log is at <MDM_HOME>/conf/log4j-jobox.properties

 

This is the configuration I was referencing, however it does not cycle with the date like you suggest. It only appends to a single log file. My original configuration is above (v6.3). Can someone share a log4j-jobox.properties configuration that does cycle daily?

 

Thanks

Employee

Re: Daily Rolling Logs for mdm-job.log

Hi,

Have a look at the log4J documentation https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html, you will have to set a date pattern option (outside the layout pattern) such:

log4j.appender.RollingAppender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RollingAppender.File=${mdm.root}/logs/mdm-job.log
log4j.appender.RollingAppender.DatePattern='.'yyyy-MM-dd
log4j.appender.RollingAppender.layout=org.apache.log4j.PatternLayout
log4j.appender.RollingAppender.layout.ConversionPattern=%d %p [%c] - %m%n

 

Five Stars

Re: Daily Rolling Logs for mdm-job.log

Thank you, this gets me a little closer; however the appender does not write out a new file each day with the date appended to the file name. It does cycle every day and recreate the mdm-job.log file. This helps keep the size of that log down, but I am missing the historical data. Here is the configuration I have in log4j.jobox.properties. What else am I missing?

 

### ====================================================================== ###
## ##
## Jobox Log4j Configuration ##
## ##
### ====================================================================== ###

log4j.rootCategory=INFO,CONSOLE,JOBFILE

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%p [%c] - %m%n

 

log4j.appender.JOBFILE=org.apache.log4j.DailyRollingFileAppender
log4j.appender.JOBFILE.File=${mdm.root}/logs/mdm-job.log
log4j.appender.JOBFILE.DatePattern='.'yyyy-MM-dd
log4j.appender.JOBFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.JOBFILE.layout.ConversionPattern=%d %p [%c] - %m%n

Employee

Re: Daily Rolling Logs for mdm-job.log

from the Log4J doc:

For example, if the File option is set to /foo/bar.log and the DatePattern set to '.'yyyy-MM-dd, on 2001-02-16 at midnight, 
the logging file /foo/bar.log will be copied to /foo/bar.log.2001-02-16 and logging for 2001-02-17 will continue in /foo/bar.log until it rolls over the next day.



So it is expected at each day new content is added from scratch to the same file. Do you mean you lost the .log.xxxx-xx-xx files from the days before?

 

Five Stars

Re: Daily Rolling Logs for mdm-job.log

Yes, the mdm-job.log is overwritten each day and I lose the mdm-job.log.xxxx.xx.xx (actually it's never created).

 

I also tried this configuration with the append declaration;

log4j.appender.JOBFILE.Append=true

 

This produced the same result.

 

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch