Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Help me understand tUniqRow's de-duplicating behavior

Highlighted
Seven Stars

Help me understand tUniqRow's de-duplicating behavior

I have some data which looks like this:

Col1     Col2      Col3      Col4

1          abc        wer        4

2          abc        wer        3

3          abc        wer        6

 

My requirement is to get a unique row, which is unique at col2 and col3 level. While I don't need what's in the Col1 or Col4, I intend to keep the highest value of Col4. This requirement thus makes me want to keep record number 3.

 

In the tUniqRow component, I only have the option to specify col2 and col3. Going by that, there are three records matching that criteria.I can't check the col4 since it'll yield three different records (pretty much the dataset).

 

How would the tUniqRow component know, which record to keep? If it 'always' picks the first record of the group, I must sort the data at Col4 level so that the record with value 6 comes at the top. However, since I don't understand how the tUniqRow operates, I can't take a call.

 

It'd be great to have someone confirm this.

 

 

Highlighted
Seven Stars

Re: Help me understand tUniqRow's de-duplicating behavior

Hello,

 

Before tuniqrow use component tsortrow to sort the column 4 in desc order.

 

tuniqrow.PNG

 

Regards

Ganshyam

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog