One Star

How to pas last modified file to FileInputDelimitted

Hi There,
I have searched all over, and read this post: knowledgeoriented. blogspot. com/2011/11/limiting-rows-in-talend-data-flows.html
But it doesn't seem complete and doesn't work.
The situation: I need to get the last modified file from a directory on the local machine. I then need to pass that file into the fileinputdelimited component.
I currently have:
tfilelist --> iterate --> titeratetoflow --> tsamplerow -->tflowtoiterate -> tinpufiledelimited ---> tlogrow (just to make sure its pulling the right file)
But it doesn't work. I have it configured so that titeratetoflow has a column called "FileName" with "((String)globalMap.get("CURRENT_FILE"))" as the value, "FileDirectory" with ((String)globalMap.get("CURRENT_FILEDIRECTORY")) as value, and "FileAndDirectory" with ((String)globalMap.get("CURRENT_FILEPATH")) as value.
The tsamplerow is limited to "1".
The tiflowtoiterate is set so that "FileNameOnly" is value of "FileName" , "FileDirectoryOnly" is "FileDirectory" and "FilePathComplete" is "FileAndDirectory"
In the File location field of the tinputfiledelimited, I have "((String)globalMap.get("FilePathComplete"))"
When it runs I get an error saying cannot find file or path. If I cut out the fileinput component and have it send straight to the tlogrow, it shows a single line of blank entry.
Any ideas?
4 REPLIES
Four Stars

Re: How to pas last modified file to FileInputDelimitted

Hi Cori,
Use tFileProperties component, you will get complete metadata of the file which also has a unix timestamp for date modified...
If you know the pattern or previous timestamp of the file, then you can compare the timestamps and based on that pick up the right file.
I think you got an idea to it.
Thanks
Vaibhav
One Star

Re: How to pas last modified file to FileInputDelimitted

Hi Vaibhav,
Thanks for the reply. I don't need the date/time. I need to have the tfileinputdelimited component select the last updated file of all files in a given directory. Will the fileproperties component help with this?
Thanks again for your help.
One Star

Re: How to pas last modified file to FileInputDelimitted

I should also mention on the tfilelist I have it set so that it sorts by last modified, descending. That was how I thought I could narrow it down to the newest file, by then having the samplerow limited to the first entry.
Four Stars

Re: How to pas last modified file to FileInputDelimitted

Hi,
Complete flow depends on How you define "last modified"?...
You can order by using date, but how you ensure that the file is modified... When you say last modified means we need some reference for validation... There could be multiple logics to identify this...
Vaibhav