List all files and put into S3 bucket

Nine Stars

List all files and put into S3 bucket

Hi Talend experts

 

I am having difficulty when I list all the files and iterate to put them in S3 Bucket.

 

I am using below job design.

 

Screen Shot 2019-09-20 at 11.49.12 am.png

I have got following things configured in tfilelist.

 

Screen Shot 2019-09-20 at 11.49.29 am.png

 

I have got following things configured in S3put component:

 

Screen Shot 2019-09-20 at 11.49.18 am.png

 

Not sure what's going wrong but it iterates successfully and job runs fine but nothing gets added to S3 bucket.

 

Can you please help me if I am doing something wrong?

 

Thanks

Harshal.


Accepted Solutions
Seven Stars

Re: List all files and put into S3 bucket

Hello,

 

I think you need the file name into the key, try this after your landedFileURI

+((String)globalMap.get("tFileList_1_CURRENT_FILE"))

 

Guillaume

View solution in original post

Seven Stars

Re: List all files and put into S3 bucket

"tFileList_2_CURRENT_FILE" in your case

View solution in original post


All Replies
Nine Stars

Re: List all files and put into S3 bucket

Screen Shot 2019-09-20 at 12.04.16 pm.png

Also when I print the used variables in S3 bucket, it shows me correct value.

Nine Stars

Re: List all files and put into S3 bucket

@rhall_2_0  , @manodwhb , @nikhilthampi  Can you please reply if you can find out something weird?

Forteen Stars

Re: List all files and put into S3 bucket

@Parikhharshal , Once job is successfully run have you refreshed bucket ? Are you verifying in the right bucket?

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Seven Stars

Re: List all files and put into S3 bucket

Hello,

 

I think you need the file name into the key, try this after your landedFileURI

+((String)globalMap.get("tFileList_1_CURRENT_FILE"))

 

Guillaume

View solution in original post

Seven Stars

Re: List all files and put into S3 bucket

"tFileList_2_CURRENT_FILE" in your case

View solution in original post

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog