I have included this line "Number of records inserted : "+((Integer)globalMap.get("tOracleOutput_3_NB_LINE_INSERTED"))" in the tSendMail component .
It gives wrong count in the email but the number of records inserted in the tables is correct.
It works fine in our staging environments but not in production.
I have turned on batch mode for the oracle output components to increase performance. not sure if thats causing the wrong count in email but we get correct count in staging environments.
You need to give more detail. Can you give us a screenshot of your job and show us where you are trying to use the count (which component)? There are a number of legitimate reasons why the counts can be wrong.
In your job, where you are using this expression ? Like post tOracleOutput_3 completion or on sub job completion & when you are claling tSendMail? We need to see flow to advice your further.. it could be because of many reason.
Also compare to staging, your target table already have data in prod? How you are counting the records and making sure its inserted by your tOracleOutput3 and not by any other process? There will be some uniqueness right?
This looks like the count should be OK. Can you tell us what sort of numbers you are getting and how many out they are? Also, can you show us the config of the tOracleOutput component?
Where is your connection component (I notice you are using a shared connection) and where do you commit your transactions? Also, what numbers are reported back and what numbers are actually inserted?
Yes i am using a shared connection and when we toracleoutput the records are auto committed we dont have to explicitly define commit in that component.
the actual number is around 200k but in the email i get around 1000 .
OK, I *think* this might be caused by the latency of the automatic commit. Can you try adding a commit component before the email component is fired? If that doesn't work, try setting the tOracleOutput component to have its own connection. I am sure that will work.
When a DB component has its own connection, it will commit while it is processing automatically. Once the component has finished, the commits have happened. So when you call on the count record in a later subjob, the total is correct. I have seen issues like this before.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.