Copy most recent file from one directory to another

One Star

Copy most recent file from one directory to another

Hi everyone.
I am trying to build a job in open studio that copies a .txt file from one location to another location. There will be a number of .txt files in the directory, but I only want to copy across the most recent one.
for example 
file1.txt
file2.txt
file3.txt
I am trying to do this based on the date modified properties.
I have looked through a few posts on here but have not managed to get any solution to work.
Would anyone be able to point me in the right direction, or even better if you have any print screen examples I could follow?
Many thanks in advance!
One Star

Re: Copy most recent file from one directory to another

Hi,
Did you try tFileList ordered by modification date desc tFileProperties (mtime and mtime_string fields give you the last modification date and time). You may also decide to break the iterate loop or jump over the undesired rows (see this post https://www.talendforge.org/forum/viewtopic.php?id=17120).
Regards,
TRF
One Star

Re: Copy most recent file from one directory to another

I have tried the solution in the attached image.
I keep getting mtime = null in the tlog though. I am clearly doing something wrong?

One Star

Re: Copy most recent file from one directory to another

Use "((String)globalMap.get("tFileList_2_CURRENT_FILEPATH"))" in tFileProperties and put a tLogRow just after this component.
I've just tested on my environment. It works like a charme.
.---------------------------------------------------------------------+------------------------------------------------+--------------------+-----------+----+-------------+----------------------------.
| tLogRow_1 |
|=--------------------------------------------------------------------+------------------------------------------------+--------------------+-----------+----+-------------+---------------------------=|
|abs_path |dirname |basename |mode_string|size|mtime |mtime_string |
|=--------------------------------------------------------------------+------------------------------------------------+--------------------+-----------+----+-------------+---------------------------=|
|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0170301.csv|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|xxxxxxxx20170301.csv|rw |103 |1488442149885|Thu Mar 02 09:09:09 CET 2017|
'---------------------------------------------------------------------+------------------------------------------------+--------------------+-----------+----+-------------+----------------------------'


Regards,
TRF
One Star

Re: Copy most recent file from one directory to another

Yes, This is working and it is now writing the most recent file properties to a tlog, which is great.
How do I use those details in the tlog to then copy the file across to a new location?
One Star

Re: Copy most recent file from one directory to another

Just have to configure a tFileCopy with the required informations
One Star

Re: Copy most recent file from one directory to another

Thank you TRF.
This is the part I am struggling with. The tlogrow only has a error message or a number of lines option to feed into the tfilecopy. Using the current filepath from the tfilelist only copies across the first file it comes to in the original directory and not the most recent file.
Apologies if this is an easy task, I have only recently picked up talend again and I am trying to get up to speed.
Regards,
One Star

Re: Copy most recent file from one directory to another

Hi,
This is not normal to have only 1 field except if tBufferInput_1 schema is not aligned.
Did you check it?
One Star

Re: Copy most recent file from one directory to another

Apologies I did make some changes to the job as I was not getting the tbuffer to work.
This is a print screen of my job. Up to the tlogrow everything appears to work as expected in that it outputs the file properties of the most recent file to the tlogrow.
You can see the options I have on the left in terms of what to use in the tfilecopy.
One Star

Re: Copy most recent file from one directory to another

OK, it doesn't work like this.
Click on tLogRow then edit schema. Here are the available fields.
So in tFileCopy, you can use abs_path to indicate which file you want to copy.
One Star

Re: Copy most recent file from one directory to another

Thanks again for your response.
I can see the abs_path in the schema of the tlogrow, but how do I use that in the tfilecopy to indicate the file to use?
One Star

Re: Copy most recent file from one directory to another

Well, for tFileCopy try row7.abs_path.
Should work.
One Star

Re: Copy most recent file from one directory to another

You my friend are an absolute star!
Do you happen to have any guidance to explain the row7 part? It is working like a charm.
Many thanks!
One Star

Re: Copy most recent file from one directory to another

Thank's Richard, it's a pleasure for me to help people as often as I can.
If you click the red arrow after tLogRow (row7), details are displayed on the component tab.
Just refere to flowname.fieldname to use them as soon as they have a value.
Carefull, if you refere to flowname.fieldname in a subjob after the one where this flow is defined, you'll get the las value for this field.
Regards,
TRF
One Star

Re: Copy most recent file from one directory to another

Great, very helpful!