How to write own numbers to tFlowMeter-Table?

Highlighted
Five Stars

How to write own numbers to tFlowMeter-Table?

I have a job where I download a bunch of PDF-Files. I get the filenames and the count of pages of each document from a DB, then I iterate over the files to process them. I would like to  write the count of files and the sum of pages to the same DB-table I use for tFlowMeter. But there are no suitable rows in the job I count set to monitored. It is easy to get the two numbers I want to log.

So far I can think of two ugly solutions:

1. Create two dummy row with tRowGenerator, set its row-number to the count to be monitored and set those rows to monitored.

2. Use tFixedFlowInput to create a Row fitting to the tFlowMeter-Schema and send it to a DB-Output. I can get most of the column values like "pid" and "job_version", only "system_pid" and "job_repository_id" I cannot get.

Is there a simpler way? Am I the only one who would like to flowmeter without a row?


Accepted Solutions
Community Manager

Re: How to write own numbers to tFlowMeter-Table?

If you want to add a row for a count, you are practically there with your second suggestion. I believe you have the important values (job_repository_id is not important). 

View solution in original post


All Replies
Community Manager

Re: How to write own numbers to tFlowMeter-Table?

While this can be done, I don't understand why you want to do it? You are trying to put your business "batch" logging alongside the Talend technical job logging. Why not log your business "batch" data in its own specific table with a link to the Talend data (using the PID)? That way you can have total control over what goes in your table, can control how often it is written to, when it is written to, under what circumstances it is written to, AND join it back to the other Talend technical logging easily.

Five Stars

Re: How to write own numbers to tFlowMeter-Table?

What I like on the technical logging tables is the reporting in AMC. The numbers I want to logg are similar to a count on a row. So I thought why no use what is already there. But I am open to arguments.

Community Manager

Re: How to write own numbers to tFlowMeter-Table?

If you want to add a row for a count, you are practically there with your second suggestion. I believe you have the important values (job_repository_id is not important). 

View solution in original post

Five Stars

Re: How to write own numbers to tFlowMeter-Table?

On second thought, after I tried it, you can write to the tFlowMeter-Table directly, but without the "job_repository_id" AMC does not show the value. So for AMC the "job_repository_id" is part of the key for a job execution and thus important. There seems to be no easy way to get that "job_repository_id". I don't want to read it from the AMC-DB.

Community Manager

Re: How to write own numbers to tFlowMeter-Table?

Click on your code tab and search the code for this....

"MetterCatcherUtils talendMeter_METTER = new MetterCatcherUtils"

 

I think the job_repository_id is shown below. You can access it using one of the talendMeter_METTER methods I believe as well. But you will need to hunt this out in the Java.

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog