Write NB_LINE_INSERTED to database

Highlighted
Six Stars

Write NB_LINE_INSERTED to database

I'm parsing a CSV file and writing it to SQL using tMSSqlOutput - this works fine.  I'd like to capture the NB_LINE_INSERTED value and insert that into an additional log file.

 

1.  I assume I should trigger  "On component OK" but I cannot "connect" that to another tMSSQLOutput control or to a tMap control.- what should me next "step" be after the database write to catch the variable and then write that to another table?  

2. Is there a best practice or tutorial on logging various jobs' activity?


Accepted Solutions
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

All Replies
Four Stars

Re: Write NB_LINE_INSERTED to database

There several ways of doing this. Not sure if there is a best practice scenario from which to work. 

 If you are looking to capture processing details of a job. I would first suggest to think about the data elements you want to capture. The way I have done this is to use the row generator ( 1 row only). 

Basically, once the job has completed, either a oncomponent or onsubjob then connects to the row generator. this captures the information I need to save to my log table (this includes to the number of rows read from source and number inserted into the destination table) (See attached pics) . This then writes out to the database on a Different SQLoutput connection. 

 

 

Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Write NB_LINE_INSERTED to database

That worked perfectly, thanks Sabrina!
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

Great it works. Thanks for your feedback.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch