Six Stars

How to exclude junk files ( file name starting with "~$") from a local directory

Yesterday, I posted a question on "How to skip temporary files (~$ excel file name) when using tFTPPut to transfer files to remote server?"

https://community.talend.com/t5/Design-and-Development/How-to-skip-temporary-files-excel-file-name-w...

 

I solved the problem with help from the community. For that problem, all the source files are in ONE single folder.

 

Now I have a more complicated issue. I want to go through the directories (folders/subfolders/files) to exclude the junk files. I don't think I could go through the original directories to just delete the bad ones. Most likely I will need to create a copy of the directories and during the process I have filter to exclude these ~$ files. Important!!! I must keep the directories unchanged (meaning I can't copy all the files from directories to one new folder), because eventually I will have to create the same directories on a remote server and transfer the valid files from local to remote with tFTPPut.

 

My difficulty here is: I am working with Directories (not only Files).

 

Please let me know if you have any idea on this. Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Nine Stars

Re: How to exclude junk files ( file name starting with "~$") from a local directory

Hi,
If you know how to filter and know what isn't junk, you also know what to delete.

>> "I don't think I could go through the original directories to just delete the bad ones."
Just filter on your previous exclude mask in tFileList , you can use glob / regex ... so you can be pretty specific on what you want to filter. So I'm pretty sure you can go through and delete them, just make sure you test it.

I suggest you first do the ftp thingy and if it was successful you delete these temp files, the tFileList you search for "~$*" and in the tFileDelete you use (String)globalMap.get("tFileList_1_CURRENT_FILEPATH") as file to delete. Look at documentation: tFileDelete Docu


Hint 1: if you use CTRL+Space you get a drop downlist where you can select variables from used components in your current job.
Hint 2: tFileList advanced settings there's an exclude option
Hint 3: Look at documentation OnSubjobOk : https://community.talend.com/t5/Design-and-Development/What-is-the-difference-between-OnSubjobOK-and...

1 REPLY
Nine Stars

Re: How to exclude junk files ( file name starting with "~$") from a local directory

Hi,
If you know how to filter and know what isn't junk, you also know what to delete.

>> "I don't think I could go through the original directories to just delete the bad ones."
Just filter on your previous exclude mask in tFileList , you can use glob / regex ... so you can be pretty specific on what you want to filter. So I'm pretty sure you can go through and delete them, just make sure you test it.

I suggest you first do the ftp thingy and if it was successful you delete these temp files, the tFileList you search for "~$*" and in the tFileDelete you use (String)globalMap.get("tFileList_1_CURRENT_FILEPATH") as file to delete. Look at documentation: tFileDelete Docu


Hint 1: if you use CTRL+Space you get a drop downlist where you can select variables from used components in your current job.
Hint 2: tFileList advanced settings there's an exclude option
Hint 3: Look at documentation OnSubjobOk : https://community.talend.com/t5/Design-and-Development/What-is-the-difference-between-OnSubjobOK-and...