Message in Talend Stats "success" even if error occures

One Star

Message in Talend Stats "success" even if error occures

Hi folks,
I build a simple Job to test Logging to Database and sending Mails on Errors.

I provocated an Error in the job: The Table where the rows should be inserted does not exsist so there is a java.sql.BatchUpdateException when the DB Output Component is executed. "Abort on error" is checked.

Now the strange thing for me is, that in the Statistics-Table the status of the Job is "Success":

The Error gets logged correctly in the Logs-Table:

Now my questions:
1. Is this the wanted behaviour? Shouldn't the status of the Job in the Statistics Table be "failure"?
2. Is it possible to append the Error Message to the E-Mail which gets send on "Subjob Error"
Kind regards!
One Star

Re: Message in Talend Stats "success" even if error occures

Hi,
Try to select the check box "Die on error" of the tProgresqlOutput"
One Star

Re: Message in Talend Stats "success" even if error occures

Hi,
Try to select the check box "Die on error" of the tProgresqlOutput"

I assume "Abbrechen bei Fehlern" is "Die on Error". It is checked...
Four Stars

Re: Message in Talend Stats "success" even if error occures

Hi machete,
Statistics table not only generates statistics for a job, but also for all the components. It is not necessary that the success message is related to complete job... pl check you would find some failure message as well..
thanks
vaibhav
Moderator

Re: Message in Talend Stats "success" even if error occures

Hi,
Statistics table records details like begin Time, End Time which is generated by any job.
Logs table records the messages from Java Exception, tDie and/or tWarn (prerequisite: there must be an error in your job).
For your use case, your job is finished successfully with Java Exception.
2. Is it possible to append the Error Message to the E-Mail which gets send on "Subjob Error"

Do you want to send error message printed on console to email? If so, here is a global variable called ((String)globalMap.get("tPostgresqlOutput_1_ERROR_MESSAGE")). You can use it in tsendmail.
Here is a related forum http://www.talendforge.org/forum/viewtopic.php?id=35043, hope it will be helpful for you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Message in Talend Stats "success" even if error occures

Hi,
Statistics table records details like begin Time, End Time which is generated by any job.
Logs table records the messages from Java Exception, tDie and/or tWarn (prerequisite: there must be an error in your job).
For your use case, your job is finished successfully with Java Exception.
2. Is it possible to append the Error Message to the E-Mail which gets send on "Subjob Error"

Do you want to send error message printed on console to email? If so, here is a global variable called ((String)globalMap.get("tPostgresqlOutput_1_ERROR_MESSAGE")). You can use it in tsendmail.
Here is a related forum http://www.talendforge.org/forum/viewtopic.php?id=35043, hope it will be helpful for you.
Best regards
Sabrina

Hi Sabrina,
that way the tMail Component would only show Error Messages of the specified component (tPostgresqlOutput_1).
But of course errors could occure somwhere else and I asked myself if there is a way to have something like a global generic error handler, which catches all errors of the job and puts them into the mail
But if I use tLogCatcher I don't get my log no longer in Database and File?! So my requirement is:
Have a job-wide error handler which is not tied to a specific component which logs Exceptions
1. In the logs Database Table
2. A log-file (in case db is not reachable)
3. Sends an E-Mail with the Exception Text
Kind regards
Kind regards!
Four Stars

Re: Message in Talend Stats "success" even if error occures

Hi,
Capture error logs to flat file in the tPreJob before starting the main job. Redirects all the contents to flat file..
Create independent job to read the file and if the contents length is> 0 send an email... Same file you can log to database table as well.
java.io.PrintStream ps = new java.io.PrintStream(new java.io.FileOutputStream(logFile));
System.setErr(ps);
using fixedflow component, you can write the contents of each execution to DB table as BLOB/Text
Thanks
Vaibhav
Moderator

Re: Message in Talend Stats "success" even if error occures

Hi,
You can use both Stat&log in job view/project setting and tlogcatcher component in your job design.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Message in Talend Stats "success" even if error occures

Hi,
You can use both Stat&log in job view/project setting and tlogcatcher component in your job design.
Best regards
Sabrina

Since I had some trouble with tLogCatcher I solved it that way:
On SubJobError I trigger tSendMail component which sends the logfile as attachment.
Just one more doubt I have: I tried to launch the job but my Database was not running. Therefor the job was cancelled with an Error (Could not connect to Database) before it could be executed. I think the databse connection for logging gets initialized before executing the job. But that way I would not recognize a problem with the job. Is there a way to catch and recognize these kinds of errors?
One Star

Re: Message in Talend Stats "success" even if error occures

Solved the main issue:
To have a "failure" in the stats log add a tDie component on "Component Failure" of the DB-Output.
Four Stars

Re: Message in Talend Stats "success" even if error occures

machete - glad you figured out the tDie trick. And yes, tlogcatcher --> tSendMail is a great combo for sending notifications for all kinds of errors.
On the issue of availability of your logging database. It is not inconceivable that a job might fail because the logging database is offline. A possible workaround would be to log to files first, and then have a separate job that moves the files to the logging/amc database. That removes one point of failure - but it does add an extra job.
One Star

Re: Message in Talend Stats "success" even if error occures

machete - glad you figured out the tDie trick. And yes, tlogcatcher --> tSendMail is a great combo for sending notifications for all kinds of errors.
On the issue of availability of your logging database. It is not inconceivable that a job might fail because the logging database is offline. A possible workaround would be to log to files first, and then have a separate job that moves the files to the logging/amc database. That removes one point of failure - but it does add an extra job.

thanks for the input!
Maybe you can help me with another point:
If I run Talend Jobs on Linux machines (in my case RHEL) as cron jobs, what's the best way to catch problems while starting the jobs. For example if there is a Heap Overflow or something while initialization that might prevent the job from starting so no logs get written at all. Is there a way to monitor/catch those problems with cron?
Kind regards