How to configure central Logging for all jobs in a project?

One Star

How to configure central Logging for all jobs in a project?

Hello guys, I think I read every single post about logging, but didn't find any clue to solve my simple task.
My task:
log.error("Error occurred!");
My Talend Version is Talend Open Studio (TOS) 5.5.2. I actually use ESB version, but I would need the same for Data Integration and others, so my question is not related to ESB.
I want to be able to use that statement in Routines, in tJava and in tJavaRow without polluting my 30-40 Jobs with tLog4j/tLog4jInit.
I want the logging configuration in a single place. Every logging library offers logging to a specific Logger like BUSINESS_LOGGER or PERFORMANCE_LOGGER. That way I will be able to redirect the logs.
My concrete problem:
I'm trying to use log4j directly in a routine:
org.apache.log4j.Logger.getLogger(MyRoutine.class).error("MyRoutne failed");
This gives me NoClassDefFoundError. I need to provide log4j.jar, but I don't want to add the jar to every single routine and job. I want to define the log4j.jar for the project.
I tried:

adding log4j in Settings->Java->Build path->User libraries
put the jar into <Studio Path>/lib/java
open Talend Modules view -> Import external jar
I restarted the Studio every time to make sure everything is reloaded. But didn't manage to resolve NoClassDefFoundError.
Then I need to put my log4j.xml on classpath I need to do it in one place also - I think putting tLog4jInit from Talend Exchange in every job is no solution.

Then I need to run/debug the job inside the Studio, and after I have exported the job, I need to be able to run it outside the studio.
The exported job should include the logging jar. I should be able to put a different log4j.xml on classpath for production.
Nice to have: if there is some infrastructure available like Talend ESB Logging System provided by karaf or by talend, I would like to use that, I would may be also prefer not to use log4j directly but slf4j or logback instead.
I think a lot of people try to solve the same problem. Thanks for helping! 
Seventeen Stars

Re: How to configure central Logging for all jobs in a project?

hi,
have a look at this talend help web page :
https://help.talend.com/pages/viewpage.action?pageId=207421454
regards
laurent
One Star

Re: How to configure central Logging for all jobs in a project?

Hi Laurent,
thanks for your answer! I already came across that site,
though it says "The log4j feature is only available in the Enterprise and Platform version of Talend Studio "
I don't have that button to activate log4j in TOS on the settings page.
Regards Anatoli
One Star

Re: How to configure central Logging for all jobs in a project?

Is there  really no solution for Talend Open Studio to use standard logging?
Moderator

Re: How to configure central Logging for all jobs in a project?

Hi,
With talend community version, there is Stat& Log feature.
Statistics log records details like begin Time, End Time which is generated by any job.(tStatCatcher)
Logs(tLogCatcher) records the messages from Java Exception, tDie and/or tWarn (prerequisite: there must be an error in your job,).
Flow Meter log records the number of rows processed in the defined flow(prerequisite: you have to use tFlowMeter in your job).
The log4j feature is not open source.
So far, talend provides community user with a free trail of Talend Enterprise for Data Integration Product.
You can download it freely from:Free Trial of Talend Data Integration
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: How to configure central Logging for all jobs in a project?

Hello Sabrina,
thanks a lot for clarifying Log&Stat feature is Enterprise only.
I'm still wondering if it is not possible to define a user library for all jobs and routines in a project.
Actually I tried to add it to user libraries, but NoClassDefFoundError persists.
 
Best Regards!
Anatoli
Moderator

Re: How to configure central Logging for all jobs in a project?

Hi,

There are two ways for you to export the external jar files:
1)Using tLibraryload component to load the jar file at the begininig of job. for example:
tLibraryLoad
   |
tJava(on tJava, use the external API)

2)Creating custom routine, import the external jar file by clicking on the rouinte name and edting the routine libraries, and then call this routine in the job.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: How to configure central Logging for all jobs in a project?

The log4j feature is not open source.

Is your "log4j" differ from the http://logging.apache.org/log4j/2.x/, which has Apache 2.0 (open source) license?