Percentage of rejected Rows

One Star

Percentage of rejected Rows

Hi,

I need to know the percentage of rejected rows for a condition and based on the result threshold should perform(or not) an action. I am not sure if Talend supports a conditional check on the whole result set. My problem is like this,

DB Rows -> Check for Null -> If more than 40% rows has nulls -> Skip operation (else perform operation)

Can someone help.

Thanks
San
Four Stars

Re: Percentage of rejected Rows

Ok, I threw something together and this should work.

Input -> tFilterRow -If Component Ok> tJava -> -RunIf>

use the tFilterRow to filter out which ones are null. In the tJava component create a parameter and have it calculate the percentage you need with ((Integer)globalMap.get("tFilterRow_1_NB_LINE_OK")) <-- This should give you the number that corresponds to what is null from tFilterRow.

Then create two seperate flows from tJava using RunIf, add expressions in the runIf's that check for the percentage. One will be > 40% the other will be < 40%.
One Star

Re: Percentage of rejected Rows

Great. It did work.

But once the Run If condition is true, I want to proceed with rest of the processing using the rejected rows of the filter component. How can we propagate this input after the condition is set to true.

DB Input -> Filter Condition -> On component Ok -> Determine Percentage -> Less than 40% -> (Process rejected Rows)


Thanks
Four Stars

Re: Percentage of rejected Rows

I don't have Talend open right now, but off the top of my head you can reroute the flow of the rejected rows to where the less than 40% flow is. If you right click on the tFilterRow, it allows you to use either the Filtered or Rejected rows...

Maybe that will work...
One Star

Re: Percentage of rejected Rows

Nope. That wont/dosent work. I am not able to connect two links (Run IF from the Java component & Reject from Filter) to the Map component. Also the flow doesn't seem right.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog