One Star

Get Latest Data File?

Is there a routine that can be written to automatically get the latest file or the file for the previous date? Just as 'getDate' allows you to retrieve the file for today automatically.

Example: Today is 20070724 and i want to pull the latest file available. Below are example files. The file I would want to return would be the ABC20070710 which is the latest file.
ABC20070623
ABC20070626
ABC20070630
ABC20070710
Tags (1)
8 REPLIES
Employee Xin
Employee

Re: Get Latest Data File?

yes, in job design->code->routine
I?ll give you a simple example
//template routine Java
package routines;
public class MyRoutine {
public static String getLastestFile(String directory) {
// read Directory and get the last File
return "ABC20070710";
}
}
Any component can use this routine, for instance, tMsBox, in the message testbox, type MyRoutine. getLastestFile ("c:/mydir/"),after run the job you?ll see the how routine is used.
This is a simple example of how to use routine, if you want to implement your case, some programming is needed
One Star

Re: Get Latest Data File?

Hi xliu,
wouldn't it be a nice feautere if tFileList exports a main-row (additional to the "iterate"), with file information :-)
So this problem wouldn't need any special programming (tFileList -> tAggregateRow).
Bye
Volker
One Star

Re: Get Latest Data File?

Hi xliu,
Is this routine applicable to Perl? I have written these jobs in Perl. Also, in the routine you have look for a specific date "ABC20070710". I want to automate the job so I find the latest file each day. In the example of files listed the "ABC20070710" was the latest file but if today a file "ABC20070717" shows up that would be the file I would want returned.
One Star

Re: Get Latest Data File?

Hello Rodney,
this solution is working only for java-generated code. Xliu please correct me, if I am wrong.
If you generat perl code, you have to code in perl. I have actual no talend software running, so I can actual give you only some simple advice:
Write a new perl-routine which returns the correct filename (like xliu describes for java)
Input: $path, Output: $fileWithPathname
I don't know how firm you are with perl.
You can get a list of the files with glob or the <>-operator (<filemask>).
Process this list and extract the relevant filename.
Possibly the following module can help you:
http://search.cpan.org/~yves/Date-Simple-3.02/lib/Date/Simple.pm
Bye
Volker
Employee Xin
Employee

Re: Get Latest Data File?

Hi,Volker Brehm
Component tFileList can't be linked to tAggregateRow with Iterate,Because tFileList does not have schema, while tAggregateRow does, so the job can not run to acheive this.
Cheers
One Star

Re: Get Latest Data File?

Hi xliu,
the link between tFileList and tAggregateRow (or any other component) was just a idea. I thought about opening a feauture request in the Bugtracker.
Bye
Volker
One Star

Re: Get Latest Data File?

Hi xliu,
Is there a routine applicable to Perl? I have written these jobs in Perl. Also, in the routine you have look for a specific date "ABC20070710". I want to automate the job so I find the latest file each day. In the example of files listed the "ABC20070710" was the latest file but if today a file "ABC20070717" shows up that would be the file I would want returned.
Employee

Re: Get Latest Data File?

Hi,
This topic will probably help you 247
Hope this helps