Write NB_LINE_INSERTED to database

Highlighted
Six Stars

Write NB_LINE_INSERTED to database

I'm parsing a CSV file and writing it to SQL using tMSSqlOutput - this works fine.  I'd like to capture the NB_LINE_INSERTED value and insert that into an additional log file.

 

1.  I assume I should trigger  "On component OK" but I cannot "connect" that to another tMSSQLOutput control or to a tMap control.- what should me next "step" be after the database write to catch the variable and then write that to another table?  

2. Is there a best practice or tutorial on logging various jobs' activity?


Accepted Solutions
Highlighted
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

View solution in original post


All Replies
Highlighted
Four Stars

Re: Write NB_LINE_INSERTED to database

There several ways of doing this. Not sure if there is a best practice scenario from which to work. 

 If you are looking to capture processing details of a job. I would first suggest to think about the data elements you want to capture. The way I have done this is to use the row generator ( 1 row only). 

Basically, once the job has completed, either a oncomponent or onsubjob then connects to the row generator. this captures the information I need to save to my log table (this includes to the number of rows read from source and number inserted into the destination table) (See attached pics) . This then writes out to the database on a Different SQLoutput connection. 

 

 

Highlighted
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

The global variables of NB_LINE_INSERTED, NB_LINE_UPDATED and NB_LINE_DELETED are available in the whole job. You can populate the number of lines of inserted, updated data by using tfixedflow component

The work flow looks like:

Main job
...
  |
onsubjobok
  |
tFixedFlowInput(input: receive the global variables.)-->t<DB>Output

tFixedFlowInput: pass the global variables to your DB

Let us know if it is OK with you.

1.png

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

View solution in original post

Highlighted
Six Stars

Re: Write NB_LINE_INSERTED to database

That worked perfectly, thanks Sabrina!
Highlighted
Moderator

Re: Write NB_LINE_INSERTED to database

Hello,

Great it works. Thanks for your feedback.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Write NB_LINE_INSERTED to database

Hi,

 

I'm trying to do the same but i get an error saying "Error in the component's properties:tFixedFlowInput_1 cannot be resolved to a variable", what I'm doing wrong?

 

please see attached file

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog