How to use a 'Last Update Date'

Four Stars

How to use a 'Last Update Date'

I'm using Talend Open Studio for Data Integration (6.4.0).

Moving data from Oracle to PostgreSQL; only want to update the PostgreSQL tables with new records from Oracle.

Previous successful run, I store the 'Last_Update_Date' in a separate table in PostgreSQL.

 

The Oracle input tables have a 'Create_Date' field, which I can compare to the PostgreSQL 'Last_Update_Date', and only keep the Oracle records which have date greater than 'Last_Update_Date'.

 

In summary, I want to:

Read Oracle tables, compare 'Create_Date' to the 'Last_Update_Date' I previously stored in a PostgreSQL table, insert only new records into the PostgreSQL tables.

 

Thanks for the help!


Accepted Solutions
Forteen Stars TRF
Forteen Stars

Re: How to use a 'Last Update Date'

Hi,
Read the 'Last_Update_Date' max value from Postgresql and store the result into a global variable.
Then reuse this global to the Oracle where clause, giving something like:
"Create_Date > to_date('" + (String)globalMap.get("yourGlobal") + "')"

TRF

All Replies
Forteen Stars TRF
Forteen Stars

Re: How to use a 'Last Update Date'

Hi,
Read the 'Last_Update_Date' max value from Postgresql and store the result into a global variable.
Then reuse this global to the Oracle where clause, giving something like:
"Create_Date > to_date('" + (String)globalMap.get("yourGlobal") + "')"

TRF
Four Stars

Re: How to use a 'Last Update Date'

Thanks! Works perfectly.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.