Unable to download latest files from remote FTP folder on hourly basis

Five Stars

Unable to download latest files from remote FTP folder on hourly basis

I want to download latest files from remote FTP server  where these files are pushed on hourly basis for whole day and after completion new folder for next day is created. There are 8 folders for last 8 days and I only want to get latest file from those 8 days on hourly basis.

I have deployed below logic but it is not working somehow and downloading all files instead of latest files for same hours when job is executed.

Capture.PNGMain jobCapture.PNGtFTPFilelist propertiesCapture.PNGtFTPFileProperties configurationCapture.PNGtSortRow PropertiesCapture.PNGtSampleRow propertiesCapture.PNGtFTPGet properties


Accepted Solutions
Seven Stars

Re: Unable to download latest files from remote FTP folder on hourly basis

I missed the point of tftpfilelist will already a iteration. you are right. 


All Replies
Seven Stars

Re: Unable to download latest files from remote FTP folder on hourly basis

Hi,

 

Please use the component called tflowtoiterate just after tsamplerow before tftpget

 

In tftpget you must be mentioning the file name in the file mask

 

row4.basename+".csv"

 

I hope this should work. If it is doesn't work please send me a screen shot after you modified the job.

 

Cheers!

Gatha

 

 

 

Ten Stars

Re: Unable to download latest files from remote FTP folder on hourly basis

I would be surprised if this design works for the case you described.  By iterating after tFTPFileList, the subsequent components execute once for every row returned.  So tFTPFileProperties passes one row to tSortRow, which has a very easy time sorting a single-row list which is then selected by tSampleRow and finally retrieved.

 

To find the latest dated file in the list, I'd probably pass the output of tFTPFileProperties to a tJavaRow and compare the current file's mtime to a value stored in a global variable.  If the variable is null or the incoming mtime is greater than the stored mtime, store the current row's values in variables.

 

Once the iteration is complete, start a new subjob that retrieves the filepath stored in the global variable set during iteration.

Seven Stars

Re: Unable to download latest files from remote FTP folder on hourly basis

I missed the point of tftpfilelist will already a iteration. you are right. 

Five Stars

Re: Unable to download latest files from remote FTP folder on hourly basis

Thanks it is working now. its great.