I am using tS3List and tS3Get to download files from S3 to local.
tS3Get ideally should download file to mentioned local location. But, this downloading with entire S3 path. I think this is wrong, as it is not same in tS3Put component and AWS cli command.
For Ex: If i download <bucketname>path/subpath/file.txt to <LocalDirectory> , i will get <localDirectory>/path/subpath/file.txt
Below are the component detail I am suing.
tS3List : This has bucket name and key-perfix(Entire directory path)
tS3Get : This has
bucket name : ((String)globalMap.get("tS3List_1_CURRENT_BUCKET"))
key : ((String)globalMap.get("tS3List_1_CURRENT_KEY"))
File : Local Directory
Have you tried use tS3Connection-->tS3Get to see if you can load your files to local without directory?
I am looking to download all the files from specific S3 folder to local. So, I need to use ts3get along with tS3list. When I do this, it copies entire directory not just file.
However, using tfileList and ts3put, I am able to copy file to S3 folder without copying directories.
Have you already tried use tS3Connection-->tS3Get to see if you can load your files to local without directory?
This issue is probably caused by tS3List component. There is a global variable "CURRENT_BUCKET" the current bucket name.
Here exists a new feature jira issue:https://jira.talendforge.org/browse/TDI-40053
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.