Six Stars

Issue with aws s3 components(ts3get)

Hi ,

   I am trying to download all files in my s3 bucket in one go.

  I am able to list all files using ts3list but not able to download all files .

  ts3get is only downloading the last file.

  Please guide me through this or this function is not applicable in talend.

Please go the below link , I am getting same issue.

https://www.talendforge.org/forum/viewtopic.php?id=45397

 

4 REPLIES
Ten Stars

Re: Issue with aws s3 components(ts3get)

If you followed the job design in the link you provided, the cause of the issue is the trigger link between tS3List and tS3Get. To get all the files found by tS3List you need to connect it to tS3Get with an Iterate link.
Seven Stars

Re: Issue with aws s3 components(ts3get)

did you understand the solution provide by cterenzi?

Please post an update we can help you if need further help
Thanks,
Sid
Mark as solution if this resolved your issue
Six Stars

Re: Issue with aws s3 components(ts3get)

I have written  a python code to download all files from my s3 bucket and running my code from tsystem component.

Six Stars

Re: Issue with aws s3 components(ts3get)

Python script to extract s3 files seem to be a great idea! share with us how you did it or personal message me if possible, Iterating the output was all you need if you are trying to download s3 files using Talend components,don't hesitate to get back to me on this, if you are still looking for any help on this as I have already developed a similar job in our project. Thanks.