File List - File properties

One Star

File List - File properties

In a pre set directory loop through the files and delete those that are older than 2 minutes.

Processing the contents of a directory was done using the tFileList step. However, it seems that it only retrieves the File Path of a file and not any of the other properties associated with a file like Size, Created Date, Last Changed etc.
Searched the help file, forum and tutorials for the keywords file, time and date. Sadly to no avail.
Now, I probably think that this can be done within TOS, but will require some custom code, ie. Java Step but I was wondering if there was a different way of doing it. Sadly I don't have the programming skills to create this kind of functionality.
Any help will be appreciated.
One Star

Re: File List - File properties

you can use the following "hack" to get more information about the file in a tJava component after a tFileList. Change variablenames if necessary.
System.out.print(files_tFileList_1.getName() + ": ");
System.out.print(files_tFileList_1.length() + "\n");

Hack, because I use a internal variable of tFileList, this may be changed in a future version.
Actual there is no Java (standard) method to get the create time of a file.

Re: File List - File properties

I think you can be interested in 984 (one year old and suspended but can be reactivated). I thought creating a specific component that would output all data (filemtime, size, owner, ...) into a data flow (a row link) and you suggest to have it in tFileList global variables.
I do still prefer the dedicated tFileProperties dedicated component because you can be interested in file properties outside tFileList. Imagin you have your file list in a database table.
I encourage you to add a note in this feature request and monitor it so that we're aware some users need it.
One Star

Re: File List - File properties

Thanks for the suggestion; sadly the file date is the most important one at present. That said, I do feel that all the other file properties will have a use as well. In this case a message has to go out if a file is older than a particular date. Granted, it's not really ETL but since the file object is already part of TOS I had thought that the extension of the file step or addition of the properties step would be appropriate.
Kind regards,
One Star

Re: File List - File properties

Sorry, my answer was not precise enough. The variable tFiles_tFileList_l is of type So you can use additional methods on this object. For example: long lastModifed().
More information can be found in the api:
One Star

Re: File List - File properties

Volker, thanks for this comment. This is indeed something I might be able to process further down the road. It is indeed a work around and it will have to be documented as such. In any case, it will allow me to do a few things that are on the fringe of ETL with the same tool. Will give this a go on my other notebook in the next couple of days.
Again, thanks for helping out.
One Star

Re: File List - File properties

Just for those interested in what I ended up doing was to add a tJavaStep after the the tFileList step and added the following line of code:
globalMap.put("tFileList_1_CURRENT_FILEDATE", files_tFileList_1.lastModified());

After this I added an tIteratToFlow step with a colum called Date and filled it with the following statement:

This will result in the Date field being populated with a long number which represents a the date in milliseconds after 1970. This date can then be used to do all sorts of comparison and calculations.
Again, thanks for responding so quickly and for all the useful information you have provided me.


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables


How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration


Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach