Four Stars

How to achieve SCD1 using Redshift Bulk component

Hi all,

 

Basically I'm trying to achieve SCD1 functionality using tRedshiftOutputBulkExec component. I'm able to develop this using tRedShiftOutput component. But this is very slow as I've a huge no.of records.

 

If we look at tMySqkBulkExec component there is an option called 'Action on Data'. But I don't see this option in tRedshiftOutputBulkExec component. Can we achieve somehow SCD1 using tRedshiftOutputBulkExec?

 

Note: I really don't want to use tRedShiftOutput component. I want to use only bulk component.

 

Thanks in advance

  • Big Data
2 REPLIES
Moderator

Re: How to achieve SCD1 using Redshift Bulk component

Hello,

Have you tried to use tRedshiftOutputBulk and tRedshiftBulkExec components instead of tRedshiftOutputBulkExec?

The tRedshiftOutputBulk and tRedshiftBulkExec components can be used together in a two step process to load data to Amazon Redshift from a delimited/CSV file on Amazon S3.

The advantage of using two separate steps is that the data can be transformed before it is loaded to Amazon Redshift.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: How to achieve SCD1 using Redshift Bulk component

Hi, thank you for your reply.

I don't see option called 'Insert or Update' on tRedShiftOutputBulk
component

Please help me how to update records using bulk component