tRedshiftBulkExec and tRedshiftOutputBulkExec

Seven Stars

tRedshiftBulkExec and tRedshiftOutputBulkExec

Hello everyone

 

can someone from the Talend development/support team clarify the following for me please.

 

Scenario:

Using TOS DI 6.4.1 free version

Using tRedshiftBulkExec and tRedshiftOutputBulkExec 

Using AWS S3 buckets to copy local GZIP data from local storage to AWS S3 for tRedshiftBulkExec and tRedshiftOutputBulkExec 

 

Question:

in above scenario do the above two components use Redshift COPY command to load data from AWS S3 into Redshift or do they use Redshift SQL INSERT statement?

 

I am asking because I have very high volume of data to load so I need optimal data load performance from Redshift.

 

Thanks


Accepted Solutions
Forteen Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

they use Redshift COPY command 

 

I know it and from error codes :-), but You can check it in "non direct form" also - https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/DznwVK27LM~ZRwlsGK2ClQ

For more information about the parameters, see http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html.
-----------

All Replies
Forteen Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

they use Redshift COPY command 

 

I know it and from error codes :-), but You can check it in "non direct form" also - https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/DznwVK27LM~ZRwlsGK2ClQ

For more information about the parameters, see http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html.
-----------
Seven Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

thanks, I suspected this much but wanted to confirm this just in case..... to avoid performance disappointment! 8^) LOL
Seven Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

and just like you I also did receive Redshift Copy errors during testing, so Talend does appear to use Redshift COPY and *not* the SQL Insert, which is a good thing!
Thanks for your replies!

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download