Five Stars

AWS S3 file properties

Hello,

 

I would like to know if there is a way to get file properties (like size, timestamp) of a file places in S3 bucket. 

 

I have tried the following but unable to get file size and timestamps. Also does this differ if the file in S3 bucket is .gz file.

 tS3 Connection

tS3FileList --iterate--> tFIleProperties --iterate--> tIterateToFlow --main--> tLogRow

Thanks,

A

  • Data Integration
Tags (1)
6 REPLIES
Moderator

Re: AWS S3 file properties

Hello,

So far, you have to retrieve your files from S3 and keep them on local firstly.

The work flow should be: tS3Connection-->tS3Get-->tFileProperties

Best regards

Sabrina

 
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: AWS S3 file properties

Thanks Sabrina for your reply. If that's the only way then I'll take it..

 

Six Stars

Re: AWS S3 file properties

As we have a client restriction not to download the file locally,canu please let us knw if there is a way other than downloading? or download/read file properties and delete in one component soo that we cant eve break this flow .ultimatley we shd nt have any control on seeing the file content.

Seven Stars

Re: AWS S3 file properties

Hi Sabrina,

tS3Connection-->tS3Get-->tFileProperties

In this approach once file gets downloaded(using ts3Get) it will have current time stamp on it. we will not get the actual time stamp which it has in S3 bucket.

how to get the time stamp of file without downloading from server?

Any inputs?

Thanks,
Sid
Thanks,
Sid
Mark as solution if this resolved your issue
Seven Stars

Re: AWS S3 file properties

Hi Folks,

 

Any suggestions here?

 

Thanks,

Sid

Thanks,
Sid
Mark as solution if this resolved your issue
Seven Stars

Re: AWS S3 file properties

Hi Guys,

 

I got the work around for this issue. It's using S3 Command line.

 

PFA job,component and console output images.

 

Thanks,

Sid

 

 

Thanks,
Sid
Mark as solution if this resolved your issue