Use file from FTP as Input

Six Stars

Use file from FTP as Input

Hi community experts,

 

Please advise me for below case. I have many questions hope I can get correct answer/inspirations.

Thank you so much!

 

I am trying to use files from FTP as input for my job design. I did some research on FTP connection but am still confused on FTP things. My basic idea is to use 2 different .csv files from different folders in FTP as Inputs and then connect with tMap. I have configured "FTP" component and "tFTPget" component.

My Problems/Questions are 

1. not sure if I need to fill in local directory in tFTPGet component because I want to get files from remote directory from FTP. but previously I downloaded files from FTP to local folder and then created Metadata and schema to connect with tMap.

Capture.JPGCapture.JPG

2.I can't use "main" to connect tFileInput2 with tmap - do I need to use Iterate? not sure difference between main/iterate

3. I configured directory in tFilelist component as below, but it seems not work. Also I don't know how to configure "Filemask".

Capture.JPG

 4. Not know how to configure tFileInput 

Capture.JPG

 

Employee

Re: Use file from FTP as Input

Hi,

 

     Since you need to compare only 2 files (1 main file and 1 lookup file), lets make the problem simple. First you download both files to local using FTPGet component one after other. The read them as shown in the figure from local folder and match it using tmap.

image.png

 

 

In the tFTPget component, there is filemask option. For both file gets from FTP location, specify the corresponding file name as filter condition. In case you are having some date or time (like main_file_21072018.txt), give the file name as "main_file*.txt" in filemask

image.png

 

 

If you need to extract multiple files for both main and lookup, then the solution will be slightly different as you may have to do the loop mechanism. But in your case, the above snippet should be sufficient.

 

    Please mark the topic as solution provided if this comment has helped to resolve your query.

 

Warm Regards,

 

Nikhil Thampi

Six Stars

Re: Use file from FTP as Input

Thank you for your advise! 

I have another 2 questions for now -

 

1. After I run, I get below error message. can you please give me some advise on that? does it mean that I did not connect successfully with FTP?

Capture.JPG

Capture.JPG 

2. The file I use in folder from FTP have timestamp after main file such as ISV_20180724053000.KRLOGN.ZZLGNT  and the file update daily which means timestamp change each day. I will use update one daily. in this case, how can I get the exact file I want?

 

Six Stars

Re: Use file from FTP as Input

Hi,

 

Can anyone help with my questions?

 

Thank you!

Forteen Stars

Re: Use file from FTP as Input

@szhou1,it was failing in the tFTPConnection,if you are sure about the FTP credentials what you used are working fine then you need to see the configuration of tFTPConnection component.

 

please check the below link to know more about tFTPConnection as well share configuration of FTP connection component.

 

https://help.talend.com/reader/WWQ40R_iTE5~~9VkUQrjgQ/ZOQS~zsIGnjmnauO7IvbSA

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now