tStatCatcher, tLogCatcher, and tMSSqlClose

Seven Stars

tStatCatcher, tLogCatcher, and tMSSqlClose

It appears that the tStatCatcher and tLogCatcher are actually run only at the end of a job -- which sort of makes sense given what they collect.  Can anyone confirm?  The problem this appears to create is, when using tMSSqlClose component to end the connection at post-job, it is getting fired before one or the other of the tStat or tLog are completed.

I would have expected that everything was completed before the tPostJob is fired.



image.pngJob design


image.pngError because connection was closed before tStatCatcher was able to write to DB

If I remove the tMSSqlClose, it runs without errors -- but then the connection is not closed, right?

image.pngNo error when tMSSqlClose removed

Ten Stars

Re: tStatCatcher, tLogCatcher, and tMSSqlClose

Confirmed! And annoying I know.
If your crash is severe (a dirty exit) , everything is gone.
Try using log4j (in community version not supported but possible, in payed it is) and have a process which actually monitors the tail of the log-file (outside this talend job).

One thing some components just write the error to console/stdout which will never end up in your tlogcatcher Smiley Wink ... Logging and provenance is horrible in Talend.
Six Stars

Re: tStatCatcher, tLogCatcher, and tMSSqlClose

We've done something very similar, but rather than writing to a database, we write to a Kafka topic. We then have a consumer that reads from the Kafka topic and pushes the log messages to the ELK stack that comes with the TAC, but you could easily write a consumer that writes them to the database of your choice.  Better still, once you have the log messages on the Kafka topic, you could write them to both an ELK setup and database at the same time.


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables


Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema


Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables