Hi community experts,
Please advise me for below case. I have many questions hope I can get correct answer/inspirations.
Thank you so much!
I am trying to use files from FTP as input for my job design. I did some research on FTP connection but am still confused on FTP things. My basic idea is to use 2 different .csv files from different folders in FTP as Inputs and then connect with tMap. I have configured "FTP" component and "tFTPget" component.
My Problems/Questions are
1. not sure if I need to fill in local directory in tFTPGet component because I want to get files from remote directory from FTP. but previously I downloaded files from FTP to local folder and then created Metadata and schema to connect with tMap.
2.I can't use "main" to connect tFileInput2 with tmap - do I need to use Iterate? not sure difference between main/iterate
3. I configured directory in tFilelist component as below, but it seems not work. Also I don't know how to configure "Filemask".
4. Not know how to configure tFileInput
Since you need to compare only 2 files (1 main file and 1 lookup file), lets make the problem simple. First you download both files to local using FTPGet component one after other. The read them as shown in the figure from local folder and match it using tmap.
In the tFTPget component, there is filemask option. For both file gets from FTP location, specify the corresponding file name as filter condition. In case you are having some date or time (like main_file_21072018.txt), give the file name as "main_file*.txt" in filemask
If you need to extract multiple files for both main and lookup, then the solution will be slightly different as you may have to do the loop mechanism. But in your case, the above snippet should be sufficient.
Please mark the topic as solution provided if this comment has helped to resolve your query.
Thank you for your advise!
I have another 2 questions for now -
1. After I run, I get below error message. can you please give me some advise on that? does it mean that I did not connect successfully with FTP?
2. The file I use in folder from FTP have timestamp after main file such as ISV_20180724053000.KRLOGN.ZZLGNT and the file update daily which means timestamp change each day. I will use update one daily. in this case, how can I get the exact file I want?
@szhou1,it was failing in the tFTPConnection,if you are sure about the FTP credentials what you used are working fine then you need to see the configuration of tFTPConnection component.
please check the below link to know more about tFTPConnection as well share configuration of FTP connection component.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema