GZIP not working

Seven Stars

GZIP not working

Hello everyone

 

Using DI version 6.4.1

 

I generate CSV file which I then GZIP using tFileArchive.

The GZ file is copied to AWS S3 to be loaded into Redshift Copy command.

 

Redshift can not unzip the files, it reports load error.

 

When I take same CSV file and use following command:

 

c:\cygwin64\bin\gzip -1 -v -k MyFile.csv

 

then if I copy the resulting GZIP file to AWS S3 then it works with Redshift.

 

The CSV file completely uncompressed can also be copied and loaded to AWS S3.

 

The problem appears to be the tFileArchive with a GZIP option (I tried all modes: best, fast and normal, same results).

 

What am I doing wrong?

thanks

 

Seven Stars

Re: GZIP not working

digging into this.

 

tRedshiftOutputBulk with GZIP option is working with AWS Redshift.

 

The GZIP with tFileArchiveOutput appears to be a bug  to me because GZIP should be = GZIP and if it works from tRedshiftBulkOutput then it should generate same GZIP file from tFileArchive too?

 

Is there a flaw in my understanding?

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now