In Talend Open Studio for Data Integraition, I'm trying to combine some fields from tStatCatcher and tFlowMeterCatcher into a single row to be logged into a database. Specifically I need the "end" message from tStats so that I can get the total execution duration and whether the job was a success or failure. From the FlowMeterCatcher I just needed the number of rows written to the database, so I put a FlowMeter right before the OracleOutput component. Normally tStatsCatcher sends a "begin" message row too, so I tried to filter this out using filterRow. After that I put a tmap to connect Flow to the Stat and output a row with the fields I need.
Here's a picture:
However, even with filtering out the "begin" message, it is still causing issues. If I make the stats branch "main" then FlowMeterCatcher sends a row immediately before any rows have been processed, so then I don't get a row count. If I make the FlowMeterCatcher the "main" row then it properly waits to get the row count, but it tries to join at the tmap with the empty "begin" row from StatsCatcher that got "filtered" out.
I realize there is probably a complex way to work around this by saving the different catchers individually and then combing them afterwards, but is there a simpler way to do thing during the job using these components?
I suppose, you need to use tJavaRow to set the same schema in both components and then use tUnite instead of tMap as bellow
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema
Pick up some tips and tricks with Context Variables