ts3BucketCreate Error

Highlighted
Eight Stars

ts3BucketCreate Error

Hi All,

I got a requirement where I want to create Folder On the RunTime using Talend. I am using ts3BucketCreate. I have attached my Path in snapshot. Now Upto BRANDEXTRACT it is been created already. After that I want to create Folder in Runtime. But job is giving error as like Bucket name must not be formatted as an IP Address. But I can manually create any folder with any Name in Cloudberry Explorer using the same s3 AccessKey and SecretKey. I got this below conventions. Is it really impossible to create Folders inside a s3 Path?

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters

Accepted Solutions
Community Manager

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 


All Replies
Eight Stars

Re: ts3BucketCreate Error

Community Manager

Re: ts3BucketCreate Error

You've given the answer below.....


@abhi90wrote:

 

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters

Take a look here as well: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

 

"You cannot create a bucket within another bucket."

Eight Stars

Re: ts3BucketCreate Error

So @rhall_2_0,

is it anyway possible to create Folder inside my desired Path in s3?

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

example is 

My Path will be like this

"Path1/Folder(will be created by Talend in Runtime)". Can this be achievable?

Community Manager

Re: ts3BucketCreate Error

Why would you want to? You can emulate folders. For example, you can have buckets name like this....

 

abhi90

abhi90.bucket1

abhi90.bucket1.subbucket-a

abhi90.bucket1.subbucket-b

abhi90.bucket2.subbucket-a

....etc

 

Technically the buckets are not within each other, but you can work with them as if they are.

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,
Did not got your point. So how are u advicing me to create folder?
Community Manager

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

I have tried with DNS naming conventions as mentioned by you using . instead of /. still getting the same error.

 

Community Manager

Re: ts3BucketCreate Error

The error "Bucket name must not be formatted as an IP Address" is an AWS error. I would contact AWS for support on this if you are not passing an IP address.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog