Insert update into mysql from csv

Four Stars

Insert update into mysql from csv

Today i installed and configured talend to source from a CSV to mysql db.


My CSV file is updated daily but many of the records are the same as the previous day. 

I would like to be able to source from my CSV file into my target mysql db adding a LastUpdateDate into the target when new records are added. The CSV does not have a date field.


I have been playing around trying to get this to work most of the day.

i used the TalendDate.getCurrentDate() in tmap_1 to a target column of LastUpdateDate but now its complaining about not having a key, which i do have configured.


A step by step best approach would be very helpful if anyone has been able to do this before.


Re: Insert update into mysql from csv

Please post your initial design and tmap setting to enable others to help you better.

Also Post the error/ problem you receive.



Four Stars

Re: Insert update into mysql from csv

There are a couple of ways of achieving this. It is dependent on a number of factors.

First What is the CSV being loaded into . Just a basic table a Dimensional table in a warehouse?

Do you need to process those records that are the same as the previous day or do you cut them from any further processing in the workflow?

How big is the CSV in terms of records ? This can affect how the UPSERT is done 


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.