Four Stars

Archive File from ftp to S3

Hi,

 

I am new to Talend Data Fabric. We have requirement to archive files from ftp to Amazon S3. Please help me on this.

 

Thanks,

Aparna

6 REPLIES
Moderator

Re: Archive File from ftp to S3

Hello,

From your job requirement, the work flow looks like:

tFTPConnection-->tFTPGet-->tFileList-->tFileInputXXX-->tS3Output

Please take a look at component references:TalendHelpCenter:FTP components and

TalendHelpCenter:Amazon S3 components

Let us know if it is OK with you.

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Archive File from ftp to S3

Hi Sabrina,

 

Actually my requirement is to copy files(as in lot of files) from FTP server and directly it to S3 and also log the time taken.

This is to be scheduled every 30 minutes.

And tS3output is not avaliable in my project Talend Data Fabric.

So please help.

 

Regards,

Aparna

Community Manager

Re: Archive File from ftp to S3

Hi Aparna
The job design looks like:
tFTPFileList--iterate--tFTPGet--oncomponentok--tS3put--oncomponentok--tFileDelete
tftpget: download the file to local system.
tS3put: put the file to S3 server from local system.
tFileDelete: delete the file from local system if needed.

Please take a look at the component documentation and learn each component mentioned here.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Archive File from ftp to S3

Hi Shong,

 

Thanks for your reply.

Is it possible to directly copy file from SFTP to S3 without placing it on local system?

 

Regards,

Aparna

Moderator

Re: Archive File from ftp to S3

Hello Aparna,

So far, talend don't support for transferring data by air. You have to place your ftp files on local system firstly and then load them to S3.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Employee Ram
Employee

Re: Archive File from ftp to S3

Hallo Aparna, I believe this scenario is possible with an ESB Mediation Route.

Can you design a Route as below and see if that works

cFtp -> cAwsS3