Logging Combine tStat and tFlowMeter

Highlighted
Four Stars

Logging Combine tStat and tFlowMeter

In Talend Open Studio for Data Integraition, I'm trying to combine some fields from tStatCatcher and tFlowMeterCatcher into a single row to be logged into a database. Specifically I need the "end" message from tStats so that I can get the total execution duration and whether the job was a success or failure. From the FlowMeterCatcher I just needed the number of rows written to the database, so I put a FlowMeter right before the OracleOutput component. Normally tStatsCatcher sends a "begin" message row too, so I tried to filter this out using filterRow. After that I put a tmap to connect Flow to the Stat and output a row with the fields I need.

 

Here's a picture:Capture.JPG

 

However, even with filtering out the "begin" message, it is still causing issues. If I make the stats branch "main" then FlowMeterCatcher sends a row immediately before any rows have been processed, so then I don't get a row count. If I make the FlowMeterCatcher the "main" row then it properly waits to get the row count, but it tries to join at the tmap with the empty "begin" row from StatsCatcher that got "filtered" out.

 

I realize there is probably a complex way to work around this by saving the different catchers individually and then combing them afterwards, but is there a simpler way to do thing during the job using these components?

Seven Stars

Re: Logging Combine tStat and tFlowMeter

I suppose, you need to use tJavaRow to set the same schema in both components and then use tUnite instead of tMap as bellow

 

1.JPG

DataTeam.pl

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now