discard rows with a number of null columns

Highlighted
Four Stars

discard rows with a number of null columns

Hi all,

 

I'm new to Talend. 

 

Here is my requirement: 

we need to discard rows of a table when the 25% of their columns is null.

 

E.g.:

         -------------------------------

            col1 | col2 | col3 | col4

         -------------------------------

row1    null  | 123  | null  | null

         -------------------------------

row2    456  | 123  | 789  | 000

 

 

row1 will be discarded 

row2 is good and pass the validation

 

The steps would be:

1) calculate the number of columns of a table

2) for every row, to check which column is null and count the number of null columns

3) if that number exceed the limit, then discard the row

 

Any suggestion on how to achieve it?

So far, I know how to get the first step, executing a SQL query.

 

For the second step I know the tMap component can be used to check rows, but then how to compare the numbers of columns dinamically? Maybe using a tJava component and writing code?

 

Thanks,

Nick

Tags (1)