ETL vs ELT with Amazon Redshift

Six Stars

ETL vs ELT with Amazon Redshift

We are in the process of researching how best to use Talend for some of our ETL jobs.


So, our initial design is to load data from S3 to a "landing" schema in Amazon Redshift using the "COPY" commands - we will use tRedshiftRow for this.


Since we will use Redshift as both our source and target databases, is it possible to push a lot of the "transformation" work down to Redshift? I know we can execute custom SQL using the tRedshiftRow component. However, if we use, say the tMap component, is it possible to execute the output of this component in Redshift instead of in the Talend job server? The only reason we want to do this is to make use of the horsepower that's available to us in Amazon Redshift.


I would appreciate feedback.

Forteen Stars

Re: ETL vs ELT with Amazon Redshift

As You describe - no, You can not.


If You use tMap - it mean - You make local for Talend transformations

You download (stream) data from Redshift to Talend, transform them and then upload (direct or to S3 for bulk loading)


You can check - where Talend Cloud locate really (i will be not surprise if on  AWS as well), or check other cloud ETL solutions, or Install Talend Server (or Jobs) on AWS if most of Your flow - Redshift to Redshift


And of course You can make some transformations on target Redshift instance - load, than transform depending what really You need from ETL or ELT process



Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences


Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables


Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now