Using DI version 6.4.1
I generate CSV file which I then GZIP using tFileArchive.
The GZ file is copied to AWS S3 to be loaded into Redshift Copy command.
Redshift can not unzip the files, it reports load error.
When I take same CSV file and use following command:
c:\cygwin64\bin\gzip -1 -v -k MyFile.csv
then if I copy the resulting GZIP file to AWS S3 then it works with Redshift.
The CSV file completely uncompressed can also be copied and loaded to AWS S3.
The problem appears to be the tFileArchive with a GZIP option (I tried all modes: best, fast and normal, same results).
What am I doing wrong?
digging into this.
tRedshiftOutputBulk with GZIP option is working with AWS Redshift.
The GZIP with tFileArchiveOutput appears to be a bug to me because GZIP should be = GZIP and if it works from tRedshiftBulkOutput then it should generate same GZIP file from tFileArchive too?
Is there a flaw in my understanding?
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema
Pick up some tips and tricks with Context Variables