Five Stars

Multiupload in AWS S3

Hi,

 

I am trying to load some data from MySQL to Redshift.

I am using tMySQLInput --> tRedshiftOutputBulkExec

 

Is there a way to enable the multiupload option of S3 in the component? The files that I am trying to load are bigger than 5GB and the jobs are failing. 

 

Thanks!

  • Data Integration
4 REPLIES
Seven Stars

Re: Multiupload in AWS S3

are You already enable compression?

 

Screen Shot 2017-05-13 at 1.48.21 AM.png

-----------
Five Stars

Re: Multiupload in AWS S3

No I hadn't used that. How does it work? It just compresses the files?

Seven Stars

Re: Multiupload in AWS S3

it compress files (on the fly), and make transfer faster

I have Jobs which transfer bigger tables and all ok, test it

-----------
Five Stars

Re: Multiupload in AWS S3

it works for some bigger files it's true Smiley Happy But still it doesnt enable the multiupload. I think it had to be configured in the S3 Config.