Four Stars

How to get latest files from folder based on last run

How to get incremental files from same folder …in talend

I have to get files based on last job start date after which are files coming those files i need process..how i can achive this.pls help me..Thank you

4 REPLIES
Nine Stars TRF
Nine Stars

Re: How to get latest files from folder based on last run

Store the last filename processed in a file (and reuse this information for next run) or rename the files after they have been processed.

TRF
Four Stars

Re: How to get latest files from folder based on last run

Hi TRF, Thanks for your response....My requirement is incremental files are coming to client server from that we need process the those files, which are newly coming from last run..

 

Ex: Yesterday 10 files came ---> we processed--

 

today 5 files came ---->total 15(yesterday 10+today 5)--from this i want to get latest files and need to process..

 

 

How we can this?

 

Please share the complete Talend Design ?

Thank you so much 

Six Stars

Re: How to get latest files from folder based on last run

Can you also send your file name format?? I am also facing same issue so,
I came up with a solution where i am getting all in tfilelist and iterate those file from tJava (compare those files and get next file , b4 that i will store last run job file name)
Can anyone give me better solution.. Thanks in Advance!!
Nine Stars TRF
Nine Stars

Re: How to get latest files from folder based on last run

The simplest (the better?) is to move (using tfilecopy) the files to a special folder (archive for example) as soon they have been processed. Using this pattern, you don't have to worry about how many files you have processed last day. Just iterate over the filelist and that's all.

TRF