Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Automatically update newly added columns

Highlighted
Five Stars

Automatically update newly added columns

Hi all,

I am looking for guidance, if the following routine can be automatized in Talend?

 

I have multiple Redshift tables, which are already containing billions of records, but sometimes I need to add new columns. All of the added columns should be NOT NULL and it must not have a default value. The workaround that we currently have, is that we implement the new columns with NOT NULL and a default value, then we are populating the column with data, after that we are creating a new table with the same columns, but without the default value, then we are loading everything from the source table to the newly created one, then we are dropping the original table, and renaming the newly created table. 

 

So the steps are the following:

 

  1. Adding new column with NOT NULL DEFAULT = something
  2. Update new column with data
  3. Create new database with same columns as the original one, but the new column are NOT NULL without default
  4. Inserting everyhing from original table to the new table
  5. Dropping the old table
  6. Renaming the new table

 

Is it possible to automatize this procedure? 

Highlighted
Moderator

Re: Automatically update newly added columns

Hello,

Could you please elaborate your case with an example with input and expected output values?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog