Five Stars

Variable with subfolders of directory

Hi guys,

I'm new to this software.

I created a job that imports values from txt files into a sql table.

In my source directory I have multiple subfolders with those txt files.

And I want to have a variable with all the subfolders and then specify in the job to execute it using the diferent folders.

Is this possible with Talend?

How can i do it?

Thanks in advance!

1 ACCEPTED SOLUTION

Accepted Solutions
Twelve Stars

Re: Variable with subfolders of directory

The tFileInputPositional should be able to be left the same regarding your config (assuming that the format of all of the files remains the same). However you will need to replace the file name/path parameter with the globalMap parameter I mentioned in my last post.

 

Your output from your job suggests that you (or the user running the Talend Studio) do not have access to the Loja_Teste and 9001622 files/folders.

Rilhia Solutions
7 REPLIES
Twelve Stars

Re: Variable with subfolders of directory

If I understand you correctly, you can use a tFileList (https://help.talend.com/reader/MjIZDoU6r0vLbCpGh5GAcg/8n6CoqqqbFSdHMxL6goOyw) to iterate through the files and folders. If you select "include sub-directories" this will find every file within a folder structure.

 

To filter which folders to retrieve files from you could load a list from a parameter file. The when a file and folder are picked up by the tFileList, use the data in the parameter file to filter it in or out.

 

If you already have a job loading the data from files within a folder, this change should not be too complex. 

Rilhia Solutions
Seven Stars

Re: Variable with subfolders of directory

Check out the tFileList Component,

https://help.talend.com/reader/MjIZDoU6r0vLbCpGh5GAcg/8n6CoqqqbFSdHMxL6goOyw

tFileList.PNG

You can use the iterate link of this component to loop through all the files in the subdirectory and load them one by one.

 

Five Stars

Re: Variable with subfolders of directory

@rhall_2_0thanks for your answer.

So, i created the tFileList as you said.

Now, on the tFileInputPositional how can i define it to use the file from each folder?

(As I said, I just started to use Talend)

Thanks again!

Twelve Stars

Re: Variable with subfolders of directory

The tFileList provides globalMap variables for you to use. If you go to the "outline" tab in the bottom left of your Studio (where lots of component names are), you will be able to expand the component names to see the variables that are available. The variable you will want to use for this will be something like ....

 

((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

You can actually drag and drop these variables from the "Outline" tab to your component parameter field.

Rilhia Solutions
Five Stars

Re: Variable with subfolders of directory

@rhall_2_0And do I have to change something on the File Positional?

I've created it based on one file on one of the sub folders. Is this  ok?


imagem.png

imagem.png

Twelve Stars

Re: Variable with subfolders of directory

The tFileInputPositional should be able to be left the same regarding your config (assuming that the format of all of the files remains the same). However you will need to replace the file name/path parameter with the globalMap parameter I mentioned in my last post.

 

Your output from your job suggests that you (or the user running the Talend Studio) do not have access to the Loja_Teste and 9001622 files/folders.

Rilhia Solutions
Five Stars

Re: Variable with subfolders of directory

Thank you very much for your help Smiley Happy