Write NB_LINE_INSERTED to database

Six Stars

Write NB_LINE_INSERTED to database

I'm parsing a CSV file and writing it to SQL using tMSSqlOutput - this works fine.  I'd like to capture the NB_LINE_INSERTED value and insert that into an additional log file.

 

1.  I assume I should trigger  "On component OK" but I cannot "connect" that to another tMSSQLOutput control or to a tMap control.- what should me next "step" be after the database write to catch the variable and then write that to another table?  

2. Is there a best practice or tutorial on logging various jobs' activity?

Tags (2)

Accepted Solutions
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

All Replies
Four Stars

Re: Write NB_LINE_INSERTED to database

There several ways of doing this. Not sure if there is a best practice scenario from which to work. 

 If you are looking to capture processing details of a job. I would first suggest to think about the data elements you want to capture. The way I have done this is to use the row generator ( 1 row only). 

Basically, once the job has completed, either a oncomponent or onsubjob then connects to the row generator. this captures the information I need to save to my log table (this includes to the number of rows read from source and number inserted into the destination table) (See attached pics) . This then writes out to the database on a Different SQLoutput connection. 

 

 

Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Write NB_LINE_INSERTED to database

That worked perfectly, thanks Sabrina!
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

Great it works. Thanks for your feedback.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.