Merging data with rules.

Highlighted
Four Stars bk
Four Stars

Merging data with rules.


Hello,

 

I would like to merge data from multiple sources with intention to create one 'golden record'.

I'm looking for some advice how to achive workflow solution that will automatically select and pass fields based on rules.

 

Example data to merge:

Data Source: 01.xml

Node ID: 001

Node Description: Lorem ipsum dolor sit amet

Node Photos: 001.jpg, 002.jpg

 

Data Source: 02.xml

Node ID: 001

Node Description: Lorem ipsum dolor sit amet, consectetur adipiscing elit

Node Photos: abc.jpg

 

Rule for Node Description field: select longest string

Rule for Node Photos: select source with more values (photos)

 

So after all my 'golden record' should be:

Node ID: 001

Node Description: Lorem ipsum dolor sit amet, consectetur adipiscing elit

Node Photos: 001.jpg, 002.jpg

 

I'm on Open Studio.

 

Regards,

Bart

Eleven Stars

Re: Merging data with rules.

my question is how many rules and do they change?

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Eleven Stars

Re: Merging data with rules.

you can do it with tmap.
first select all possible ids
then use ids as main input with lookup on sources.
add var values for calculation and use inline if (condition)?true:false into output.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Cloud Free Trial

Try Talend Cloud free for 30 days.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.