Hello Talend Users,
I have a FTP directories with many subdirectories (representing years from 1999 to 2018) . For now I can only download all files in all folders. However I want to only download the folders from 2011 to 2018.
So far my Job looks like this:
How can I iterate over the Folders from 2011 to 2018? With tForeach? And if yes how can I connect it to FTPGet?
Thanks in Advance.
Do you want to list sub-directories in FTP folder?
Could you please let us know if this resolved topichttps://community.talend.com/t5/Design-and-Development/Exclude-File-Mask-in-TFTPFileList/m-p/124806 is OK with you?
This is not really the solution to my problem as it is about excluding certain files from the download. I however want to download only the directories 2011 to 2018 in a list of directories which reaches from 1999 to 2018. So somehow I have to provide a list to the FTPget job.
The FTP Directory looks like this:
You could use tSSH to filter out the folders using unix commands. The output of tSSH you could then store in global variable which is tSSH_1_STDOUT
Once you have the list of folders in the global variable just use them to iterate over using
tFTPFileList -> iterate ->tFTPGet
Sample unix command to filter out folders starting with "talend" -
"find $(pwd) -maxdepth 1 -type d -name \"talend*\""
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Accelerate your data lake projects with an agile approach