function to get values occurrence number

Highlighted
Five Stars

function to get values occurrence number

Hello,

 

In Data Preparation, I would like to create a column with the number of occurrences of the values of another column. This would filter the duplicate values for example.
I didn't find out how to do that.

 

Lilian

Lilian Prud'homme
Talend ETL consultant
Highlighted
Employee

Re: function to get values occurrence number

Hello,

I am actually not sure why you want to create a new column. As you describe it, the profiling panel of Data Prep seems to be the perfect candidate for such a use case. You can see very easily the number of occurrence of your values (so that you can directly see duplicates and unique values) and then you can directly filter by selecting them.

 

See the attached screenshot where you can see 2 duplicate values and 1 unique value directly in the profiling bar chart.

 

Regards,

Patrick

Highlighted
Five Stars

Re: function to get values occurrence number

Hello Patrick,

You're right, profiling panel allows to visualize the duplicates and filter them but nothing allows to filter all duplicated values via an action.
For example: I have the following data:
col_1
-------
1
1
2
3
3
3
4

I wish I could apply an action on col_1 to calculate the number of occurrences of values as follows:
col_1    col_2
-------    -------
1          2
1          2
2          1
3          3
3          3
3          3
4          1

Then, I could apply an action on col_2 to filter the values > 1. So, automatically, I could quickly treat ALL duplicated values.

I hope my explanations are clear ;-)

 

Regards,

Lilian

Lilian Prud'homme
Talend ETL consultant

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Downloads and Trials

Test drive Talend's enterprise products.

Downloads