ts3BucketCreate Error

Eight Stars

ts3BucketCreate Error

Hi All,

I got a requirement where I want to create Folder On the RunTime using Talend. I am using ts3BucketCreate. I have attached my Path in snapshot. Now Upto BRANDEXTRACT it is been created already. After that I want to create Folder in Runtime. But job is giving error as like Bucket name must not be formatted as an IP Address. But I can manually create any folder with any Name in Cloudberry Explorer using the same s3 AccessKey and SecretKey. I got this below conventions. Is it really impossible to create Folders inside a s3 Path?

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters

Accepted Solutions
Sixteen Stars

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 


All Replies
Eight Stars

Re: ts3BucketCreate Error

Sixteen Stars

Re: ts3BucketCreate Error

You've given the answer below.....


@abhi90wrote:

 

  • Bucket names should not contain underscores
  • Bucket names should be between 3 and 63 characters long
  • Bucket names should not end with a dash
  • Bucket names cannot contain adjacent periods
  • Bucket names cannot contain dashes next to periods (e.g., 'my-.bucket.com' and 'my.-bucket' are invalid)
  • Bucket names cannot contain uppercase characters

Take a look here as well: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

 

"You cannot create a bucket within another bucket."

Eight Stars

Re: ts3BucketCreate Error

So @rhall_2_0,

is it anyway possible to create Folder inside my desired Path in s3?

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

example is 

My Path will be like this

"Path1/Folder(will be created by Talend in Runtime)". Can this be achievable?

Sixteen Stars

Re: ts3BucketCreate Error

Why would you want to? You can emulate folders. For example, you can have buckets name like this....

 

abhi90

abhi90.bucket1

abhi90.bucket1.subbucket-a

abhi90.bucket1.subbucket-b

abhi90.bucket2.subbucket-a

....etc

 

Technically the buckets are not within each other, but you can work with them as if they are.

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,
Did not got your point. So how are u advicing me to create folder?
Sixteen Stars

Re: ts3BucketCreate Error

There is no concept of a folder in S3. However you can emulate a folder structure as I described above. I'm not sure about using "/" and when I have worked with S3 I have used DNS naming conventions (see above). 

Eight Stars

Re: ts3BucketCreate Error

Hi @rhall_2_0,

I have tried with DNS naming conventions as mentioned by you using . instead of /. still getting the same error.

 

Sixteen Stars

Re: ts3BucketCreate Error

The error "Bucket name must not be formatted as an IP Address" is an AWS error. I would contact AWS for support on this if you are not passing an IP address.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.