Six Stars

Exclude File Mask in TFTPFileList

Hello,

 

I am using the TFTPFileList component, but unlike the TFileList component which has exclude file mask option in advance settings, I do not see this option in the TFTPFileList component. I want to exclude all the "xml" files and files with certain substrings that I am transferring from my remote file system to Hbase. Is there a way I can do this without storing any files or scripts on my local computer through which Talend is running? I would appreciate any help asap!

 

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions
Seven Stars

Re: Exclude File Mask in TFTPFileList

One way of achieving this would be to use Java code to filter out the files which you don't want to process in your subsequent steps.

Code for tJava -

String s = ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")).substring(StringHandling.INDEX( ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")),".")).trim();
if(!s.equals(".xml")){
System.out.println(s);
context.fileName = ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE"));
globalMap.put("tFTPFileProperties_2_FILENAME", null);
}

The above could would make sure that the immediate next component of tJava (in my example (tFTPFileProperties) won't receive the any xml file as part of it's iteration.

 

Sample job flow - 

tFTP-2.JPG

 

tFTPFileProperties settings - 

tFTP-3.JPG

3 REPLIES
Six Stars

Re: Exclude File Mask in TFTPFileList

By the way I have a large number of files of different formats, and so I would really like to exclude just the xml files then to include all the 100 other files with different formats.

Seven Stars

Re: Exclude File Mask in TFTPFileList

One way of achieving this would be to use Java code to filter out the files which you don't want to process in your subsequent steps.

Code for tJava -

String s = ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")).substring(StringHandling.INDEX( ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE")),".")).trim();
if(!s.equals(".xml")){
System.out.println(s);
context.fileName = ((String)globalMap.get("tFTPFileList_1_CURRENT_FILE"));
globalMap.put("tFTPFileProperties_2_FILENAME", null);
}

The above could would make sure that the immediate next component of tJava (in my example (tFTPFileProperties) won't receive the any xml file as part of it's iteration.

 

Sample job flow - 

tFTP-2.JPG

 

tFTPFileProperties settings - 

tFTP-3.JPG

Six Stars

Re: Exclude File Mask in TFTPFileList

Thank You for your reply. I did figure it out by trial error. I guess I should have posted my solution earlier as well. I basically did the same as you illustrated in your response, however the only difference is: you did:

System.out.println(s);

 whereas I just:

continue;

as I did not want anything to print out. However, thank you for your response as this not only just confirms my trial and error method of excluding files but can help others in the future.