Only half of the output row form my ETL job are successful inserted in MySQL db

Highlighted
Four Stars

Only half of the output row form my ETL job are successful inserted in MySQL db

Hi all, this is my frist post there.

I am new with Talend and i am not confident with English at all, so sorry for all my mistakes.

At my frist work experience they told to me to fix some error on an already existing ETL job. 

It is very complex and huge but i was able to fix some minor issue.

Now i am facing with something that i can not understand.

The job extract data form a db, do a lot of elaboration, and then save them on another db (MySQL). The problem is that not all the row that my job does after the elaboration are saved on the db, it save only half of them (more or less). My tDBOutput component receives like 22k rows (22203)  but on my db are inserted like 11k rows (11611). I dont think is a problem related only on my config, because i have the same issue on both production and test environment. The tDBoutput component is set to "create a table if does not exist" and "insert or update"  I enclose a screen shot.  Thanks 

 

 

 

 


Accepted Solutions
Six Stars

Re: Only half of the output row form my ETL job are successful inserted in MySQL db

You must have duplicate with same key after tJoin.
Regards
Abhishek KUMAR

All Replies
Six Stars

Re: Only half of the output row form my ETL job are successful inserted in MySQL db

"insert or update" option will insert if record with primary doesnot exist otherwise it will update existing record in table
Regards
Abhishek KUMAR
Four Stars

Re: Only half of the output row form my ETL job are successful inserted in MySQL db

Thanks for the answer akumar2301That is the point, the job should insert all my 22k row. It doesn't matter if the row are updated or inserted, i should have anyway all of them in my DB. I tried  to set the tDBOutput component to create a new table too,  but  nothing changed.

Six Stars

Re: Only half of the output row form my ETL job are successful inserted in MySQL db

You must have duplicate with same key after tJoin.
Regards
Abhishek KUMAR

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.