tOracleOutput_1_NB_LINE_INSERTED not accurate counting insert records

One Star

tOracleOutput_1_NB_LINE_INSERTED not accurate counting insert records

In a tJava, (after an onSubJobOk trigger linked from tPostgresqlInput_3, I record the counts in a batch (header) transaction table, based on the following:

context.QueryRecordCount = (Integer)globalMap.get("tPostgresqlInput_3_NB_LINE");

if ( (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED") != null) {
context.InsertRecordCount = (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED");
}


Now, the results:

context.InsertRecordCount , from tOracleOutput_1_NB_LINE_INSERTED is always less than the ACTUAL records inserted into the detail transaction table, which equals context.QueryRecordCount from tPostgresqlInput_3_NB_LINE. There are no filters in the output, and no errors during inserts.


Ran several tests on the same data (truncate prior to each run) - tOracleOutput_1_NB_LINE_INSERTED seems to change, but always below tPostgresqlInput_3_NB_LINE.

Once again, the actual record count saved to the tables are correct, so looks to me like tOracleOutput_1_NB_LINE_INSERTED is not accurate, and thus looks like a bug.

Screenshot of job attached.

Interesting update of findings: Going from Postgres to Oracle always yields innaccurate results, but going from Otacle to Oracle always yields accurate results.


Dave
Community Manager

Re: tOracleOutput_1_NB_LINE_INSERTED not accurate counting insert records

Hello
I tested it on TOS3.2.3 and it works fine.
There are no filters in the output, and no errors during inserts.

1)Open the oracle table and see how many rows and if all the records are inserted.
if ( (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED") != null) {
context.InsertRecordCount = (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED");
}

change to code to:
if ( (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED") != 0) {
context.InsertRecordCount = (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED");

Best regards

shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: tOracleOutput_1_NB_LINE_INSERTED not accurate counting insert records

shong wrote:
Hello
I tested it on TOS3.2.3 and it works fine.
There are no filters in the output, and no errors during inserts.

1)Open the oracle table and see how many rows and if all the records are inserted.
if ( (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED") != null) {
context.InsertRecordCount = (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED");
}

change to code to:
if ( (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED") != 0) {
context.InsertRecordCount = (Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED");

Best regards

shong

Sorry - that's not the fix - I think we have a bug here - please read the whole post.

Changed to your code, and not fixed. Again, this is downstream of tOracleOutput_1, via an okSubJobOk link - it's not an issue of null vs 0.

Essentially, the tOracleOutput does not count its inserts correctly into the talend built in record counters (although records get inserted correctly in oracle) when data originates from a tPostgresqlInput. Looks like a bug to me, until proven otherwise.

Dave

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog