Four Stars

Many filters with one condition each or one filter with many conditions which gives better performance?

Hi,

 

I have 5 filter conditions need to perform on tfilterrow. Now is which gives better performance?

 

1. Giving all the conditions in one tfilterrow?

1.PNG

or

2. giving one tfilter row for one column condition like I need 5 filters for 5 conditions?

tfilter-->tfilter-->tfilter-- like this.

 

 

 

  • Talend Integration Cloud
1 ACCEPTED SOLUTION

Accepted Solutions
Eleven Stars

Re: Many filters with one condition each or one filter with many conditions which gives better performance?

Good point about the order @TRF. Again this is an assumption, but since the tMap variables are strictly actioned from top to bottom (this I know), I would predict that the tFilterRow filters are processed in the same order. It would be interesting to test....but I reckon you would need a lot of data before you saw a repetitive difference between the order permutations.

Rilhia Solutions
3 REPLIES
Eleven Stars

Re: Many filters with one condition each or one filter with many conditions which gives better performance?

Although I have not tested this, it is almost certainly using one tFilterRow component. The individual filters will all have to be run but by setting up a tFilterRow for each you are adding unnecessary initialisation latency.

Rilhia Solutions
Nine Stars TRF
Nine Stars

Re: Many filters with one condition each or one filter with many conditions which gives better performance?

I agree with @rhall_2_0.

May having the most restrictive filter first would be a little better, but in fact it depends how the code is generated.

You may check this if you display the source code in the Studio.


TRF
Eleven Stars

Re: Many filters with one condition each or one filter with many conditions which gives better performance?

Good point about the order @TRF. Again this is an assumption, but since the tMap variables are strictly actioned from top to bottom (this I know), I would predict that the tFilterRow filters are processed in the same order. It would be interesting to test....but I reckon you would need a lot of data before you saw a repetitive difference between the order permutations.

Rilhia Solutions