GZIP not working

Highlighted
Seven Stars

GZIP not working

Hello everyone

 

Using DI version 6.4.1

 

I generate CSV file which I then GZIP using tFileArchive.

The GZ file is copied to AWS S3 to be loaded into Redshift Copy command.

 

Redshift can not unzip the files, it reports load error.

 

When I take same CSV file and use following command:

 

c:\cygwin64\bin\gzip -1 -v -k MyFile.csv

 

then if I copy the resulting GZIP file to AWS S3 then it works with Redshift.

 

The CSV file completely uncompressed can also be copied and loaded to AWS S3.

 

The problem appears to be the tFileArchive with a GZIP option (I tried all modes: best, fast and normal, same results).

 

What am I doing wrong?

thanks

 

Seven Stars

Re: GZIP not working

digging into this.

 

tRedshiftOutputBulk with GZIP option is working with AWS Redshift.

 

The GZIP with tFileArchiveOutput appears to be a bug  to me because GZIP should be = GZIP and if it works from tRedshiftBulkOutput then it should generate same GZIP file from tFileArchive too?

 

Is there a flaw in my understanding?

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog