List all files and put into S3 bucket

Highlighted
Nine Stars

List all files and put into S3 bucket

Hi Talend experts

 

I am having difficulty when I list all the files and iterate to put them in S3 Bucket.

 

I am using below job design.

 

Screen Shot 2019-09-20 at 11.49.12 am.png

I have got following things configured in tfilelist.

 

Screen Shot 2019-09-20 at 11.49.29 am.png

 

I have got following things configured in S3put component:

 

Screen Shot 2019-09-20 at 11.49.18 am.png

 

Not sure what's going wrong but it iterates successfully and job runs fine but nothing gets added to S3 bucket.

 

Can you please help me if I am doing something wrong?

 

Thanks

Harshal.


Accepted Solutions
Seven Stars

Re: List all files and put into S3 bucket

Hello,

 

I think you need the file name into the key, try this after your landedFileURI

+((String)globalMap.get("tFileList_1_CURRENT_FILE"))

 

Guillaume

View solution in original post

Seven Stars

Re: List all files and put into S3 bucket

"tFileList_2_CURRENT_FILE" in your case

View solution in original post


All Replies
Nine Stars

Re: List all files and put into S3 bucket

Screen Shot 2019-09-20 at 12.04.16 pm.png

Also when I print the used variables in S3 bucket, it shows me correct value.

Nine Stars

Re: List all files and put into S3 bucket

@rhall_2_0  , @manodwhb , @nikhilthampi  Can you please reply if you can find out something weird?

Forteen Stars

Re: List all files and put into S3 bucket

@Parikhharshal , Once job is successfully run have you refreshed bucket ? Are you verifying in the right bucket?

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Seven Stars

Re: List all files and put into S3 bucket

Hello,

 

I think you need the file name into the key, try this after your landedFileURI

+((String)globalMap.get("tFileList_1_CURRENT_FILE"))

 

Guillaume

View solution in original post

Seven Stars

Re: List all files and put into S3 bucket

"tFileList_2_CURRENT_FILE" in your case

View solution in original post

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog