I am using TOS DI 6.3.
Is there a way to do use the Talend tS3List component to do a recursive list of objects in an Amazon S3 bucket?
I am able to get a list of object under a defined prefix in the component, but if objects are added into additional deeper levels, those new objects are not being detected.
-> this is fine when listing to the folder2 prefix.
-> the random folder and files therein are not being detected by ts3List.
I would prefer not to have to develop using the s3 CLI --recursive option to get a real-time list.
As I never know what the "random_folder" may be, I'd like to emulate the --recursive function in Talend to detect new files in s3.
I do not see any --recursive option for tS3List in Talend. Any suggestions?
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema