tS3get - Issue- Loading file to local including directory

Four Stars

tS3get - Issue- Loading file to local including directory

Hi,

 

I am using tS3List and tS3Get to download files from S3 to local.

tS3Get ideally should download file to mentioned local location. But, this downloading with entire S3 path. I think this is wrong, as it is not same in tS3Put component and AWS cli command.

 

For Ex: If i download <bucketname>path/subpath/file.txt to <LocalDirectory> , i will get <localDirectory>/path/subpath/file.txt

 

 

Below are the component detail I am suing.

tS3List : This has bucket name and key-perfix(Entire directory path)

 

tS3Get : This has 

bucket name : ((String)globalMap.get("tS3List_1_CURRENT_BUCKET"))

key : ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

File : Local Directory

Tags (1)
Moderator

Re: tS3get - Issue- Loading file to local including directory

Hello,

Have you tried use tS3Connection-->tS3Get to see if you can load your files to local without directory?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: tS3get - Issue- Loading file to local including directory

Hi Sabrina,

 

I am looking to download all the files from specific S3 folder to local. So, I need to use ts3get along with tS3list.  When I do this, it copies entire directory not just file. 

However, using tfileList and ts3put, I am able to copy file to S3 folder without copying directories. 

Moderator

Re: tS3get - Issue- Loading file to local including directory

Hello,

Have you already tried use tS3Connection-->tS3Get to see if you can load your files to local without directory?

This issue is probably caused by tS3List component. There is a global variable "CURRENT_BUCKET" the current bucket name.

Here exists a new feature jira issue:https://jira.talendforge.org/browse/TDI-40053

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch