Eight Stars

ts3BucketCreate Error

Hi All,

I got a requirement where I want to create Folder On the RunTime using Talend. I am using ts3BucketCreate. I have attached my Path in snapshot. Now Upto BRANDEXTRACT it is been created already. After that I want to create Folder in Runtime. But job is giving error as like Bucket name must not be formatted as an IP Address. But I can manually create any folder with any Name in Cloudberry Explorer using the same s3 AccessKey and SecretKey. I got this below conventions. Is it really impossible to create Folders inside a s3 Path?

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters
1 ACCEPTED SOLUTION

Accepted Solutions
Fifteen Stars

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 

Rilhia Solutions
9 REPLIES
Eight Stars

Re: ts3BucketCreate Error

Fifteen Stars

Re: ts3BucketCreate Error

You've given the answer below.....


@abhi90wrote:

 

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters

Take a look here as well: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

 

"You cannot create a bucket within another bucket."

Rilhia Solutions
Eight Stars

Re: ts3BucketCreate Error

So @rhall_2_0,

is it anyway possible to create Folder inside my desired Path in s3?

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

example is 

My Path will be like this

"Path1/Folder(will be created by Talend in Runtime)". Can this be achievable?

Fifteen Stars

Re: ts3BucketCreate Error

Why would you want to? You can emulate folders. For example, you can have buckets name like this....

 

abhi90

abhi90.bucket1

abhi90.bucket1.subbucket-a

abhi90.bucket1.subbucket-b

abhi90.bucket2.subbucket-a

....etc

 

Technically the buckets are not within each other, but you can work with them as if they are.

Rilhia Solutions
Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,
Did not got your point. So how are u advicing me to create folder?
Fifteen Stars

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 

Rilhia Solutions
Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

I have tried with DNS naming conventions as mentioned by you using . instead of /. still getting the same error.

 

Fifteen Stars

Re: ts3BucketCreate Error

The error "Bucket name must not be formatted as an IP Address" is an AWS error. I would contact AWS for support on this if you are not passing an IP address.

Rilhia Solutions