Need to update date if row changes in file

Five Stars

Need to update date if row changes in file

Hi,

below is my input data:

id,name
1,ramu
2,rah
3,afaAaa


output data for 1st load:
id,name,created_date,modified_date
1,ramu,18-07-2019,18-07-2019
2,rah,18-07-2019,18-07-2019
3,afaAaa,18-07-2019,18-07-2019


My input data have some modifications(1 row update and 2 rows insert) as below

id,name
1,ramu
2,rah
3,afa
4,SADASE
5,akda


So output should be as below:
id,name,created_date,modified_date
1,ramu,18-07-2019,18-07-2019
2,rah,18-07-2019,18-07-2019
3,afa,18-07-2019,19-07-2019
4,SADASE,19-07-2019,19-07-2019
5,akda,19-07-2019,19-07-2019


The rows which are not changed/updated should not have in the change of modified_date.
I have done this by taking lookup of output table with input file which is taking very long time.
Can you please help on alternate solution

 

Thanks,

Ram

Employee

Re: Need to update date if row changes in file

Hi Ram,

 

    Since you already know the ids of the lookup data, do not pull the entire data of the table in lookup. First do an insert of new data to a temp table. For the lookup, do an inner join of the id of temp table with your lookup table so that you are picking only matched records instead of full table.

 

     You can do it my adding join conditions in the SQL query of tDBinput for lookup. In the tMap join, use left out join for the main flow with your new lookup and add the condition that if the data id is present in lookup, use the date from lookup else add current system date. This should resolve your issue. If the issue persists, please share the screenshot with these changes and error and we will take it forward from there.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Five Stars

Re: Need to update date if row changes in file

Hi Nikhil,
Thanks for your reply.
As said I already Donne that thing,which will take more time in joining if records are more.

Can you please provide any alternate solution that only modified date will only change if old records get updated

Thanks,
Ram
Employee

Re: Need to update date if row changes in file

Hi,

 

    What is the input volume and how much time its currently taking for the query? Could you please share the execution plan details if the query is taking more time? 

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Downloads and Trials

Test drive Talend's enterprise products.

Downloads