Hello Talend Users,
I have a FTP directories with many subdirectories (representing years from 1999 to 2018) . For now I can only download all files in all folders. However I want to only download the folders from 2011 to 2018.
So far my Job looks like this:
How can I iterate over the Folders from 2011 to 2018? With tForeach? And if yes how can I connect it to FTPGet?
Thanks in Advance.
Do you want to list sub-directories in FTP folder?
Could you please let us know if this resolved topichttps://community.talend.com/t5/Design-and-Development/Exclude-File-Mask-in-TFTPFileList/m-p/124806 is OK with you?
This is not really the solution to my problem as it is about excluding certain files from the download. I however want to download only the directories 2011 to 2018 in a list of directories which reaches from 1999 to 2018. So somehow I have to provide a list to the FTPget job.
The FTP Directory looks like this:
You could use tSSH to filter out the folders using unix commands. The output of tSSH you could then store in global variable which is tSSH_1_STDOUT
Once you have the list of folders in the global variable just use them to iterate over using
tFTPFileList -> iterate ->tFTPGet
Sample unix command to filter out folders starting with "talend" -
"find $(pwd) -maxdepth 1 -type d -name \"talend*\""
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema