Get the second latest file from the local drive to SFTP

Highlighted
Five Stars

Get the second latest file from the local drive to SFTP

Hi,

I need to move the second latest file in to a SFTP.  It is a process in place that generate a  report  every day except weekends. So 

to me always need to find the second latest file to move to the SFTP. For sample,  Assuming that today is Monday... I should copy last Friday file

so 

Lastest File 1 : Clients_08052019.csv   - Monday Todays Day

Lastest File 2 : Clients_08022019.csv  -- Friday

Lastest File 1 : Clients_08012019.csv-- Thursday

                 ...............

Lastest File n : Clients_mmddyyyy.csv -- Any day except weekends

 

Im using a tFileList  and using order by Modified day and Order Action Asc  but then I dont know more what to do.....

is any way I can use a Rank function here so I can always pick the Second latest file?

Thank you for all your help


Accepted Solutions
Nine Stars JR
Nine Stars

Re: Get the second latest file from the local drive to SFTP

Hi

Please try this:

1. Initialize an empty global variable via tSetGlobalVar (e.g. "file")

2. Use a tLoop (From: 1, To: 2, Step: 1)

3. -- Iterate --> tFileList (order by file name descending)

4. -- RunIf (((Integer)globalMap.get("tLoop_1_CURRENT_VALUE")) == 2) --> tSetGlobalVar (set the same variable as in the previous one to ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")))

5. From tLoop -- OnSubJobOK --> tFileInputXXX (use ((String)globalMap.get("file")) as your filename) and do whatever you like afterwards.

(look at the attached screenshot)

 

UPDATE: I played around with this a little and it does not work. Stay tuned - I am interested in this...

Nine Stars JR
Nine Stars

Re: Get the second latest file from the local drive to SFTP

OK. I have found a solution that does not require any coding. I am sure, it would be shorter it you introduce coding into this (e.g. with tJavaRow) but this works. Please find the job design attached in numerous files (zipped). You will need to include the tHashXXX components into your project (File --> Edit Project Properties --> Designer --> Palette Settings --> Technical, then mark both tHashXXX components and move them to the right via the arrow in the middle). The last tJava component is just there to output the name of the second oldest file. Replace thius e.g. with a tFileInputXXX component and use the global variable as the file name.

I hope this helps.


All Replies
Nine Stars JR
Nine Stars

Re: Get the second latest file from the local drive to SFTP

Hi

Please try this:

1. Initialize an empty global variable via tSetGlobalVar (e.g. "file")

2. Use a tLoop (From: 1, To: 2, Step: 1)

3. -- Iterate --> tFileList (order by file name descending)

4. -- RunIf (((Integer)globalMap.get("tLoop_1_CURRENT_VALUE")) == 2) --> tSetGlobalVar (set the same variable as in the previous one to ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")))

5. From tLoop -- OnSubJobOK --> tFileInputXXX (use ((String)globalMap.get("file")) as your filename) and do whatever you like afterwards.

(look at the attached screenshot)

 

UPDATE: I played around with this a little and it does not work. Stay tuned - I am interested in this...

Nine Stars JR
Nine Stars

Re: Get the second latest file from the local drive to SFTP

OK. I have found a solution that does not require any coding. I am sure, it would be shorter it you introduce coding into this (e.g. with tJavaRow) but this works. Please find the job design attached in numerous files (zipped). You will need to include the tHashXXX components into your project (File --> Edit Project Properties --> Designer --> Palette Settings --> Technical, then mark both tHashXXX components and move them to the right via the arrow in the middle). The last tJava component is just there to output the name of the second oldest file. Replace thius e.g. with a tFileInputXXX component and use the global variable as the file name.

I hope this helps.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog