1. Create Context which saves MAX (surrogate_key) from Dimension table,
2. Create variable for Surrogate_Key ,
Variable (SK) = Numeric.sequence("COVER_CLASS_SK",(Integer)context.COVER_CLASS_SK + 1,1) ,
3. I understand running on multiple executors (Spark framework) the numeric sequence won't work,
4. Can someone please suggest with an example how to go about generating sequence numbers for dimension Surrogate keys?
just to keep it here as possible answer. Because it is just a surrogate key UUID could work good in this case.
it could be:
in case of primary key we do not need sequenced incremental, just must be unique
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to make your data more available, reduce costs and cut your build time
Read about OTTO's experiences with Big Data and Personalized Experiences
Take a look at this video about Talend Integration with Databricks