Six Stars

tRedshiftBulkExec and tRedshiftOutputBulkExec

Hello everyone

 

can someone from the Talend development/support team clarify the following for me please.

 

Scenario:

Using TOS DI 6.4.1 free version

Using tRedshiftBulkExec and tRedshiftOutputBulkExec 

Using AWS S3 buckets to copy local GZIP data from local storage to AWS S3 for tRedshiftBulkExec and tRedshiftOutputBulkExec 

 

Question:

in above scenario do the above two components use Redshift COPY command to load data from AWS S3 into Redshift or do they use Redshift SQL INSERT statement?

 

I am asking because I have very high volume of data to load so I need optimal data load performance from Redshift.

 

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Nine Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

they use Redshift COPY command 

 

I know it and from error codes :-), but You can check it in "non direct form" also - https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/DznwVK27LM~ZRwlsGK2ClQ

For more information about the parameters, see http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html.
-----------
3 REPLIES
Nine Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

they use Redshift COPY command 

 

I know it and from error codes :-), but You can check it in "non direct form" also - https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/DznwVK27LM~ZRwlsGK2ClQ

For more information about the parameters, see http://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html.
-----------
Six Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

thanks, I suspected this much but wanted to confirm this just in case..... to avoid performance disappointment! 8^) LOL
Six Stars

Re: tRedshiftBulkExec and tRedshiftOutputBulkExec

and just like you I also did receive Redshift Copy errors during testing, so Talend does appear to use Redshift COPY and *not* the SQL Insert, which is a good thing!
Thanks for your replies!