tMap filter expression not working?

One Star

tMap filter expression not working?

Good Day All,
Any help would be greatly appreciated!
MY REQUIREMENT:
I am extracting data from our database into an excel report. Unfortunately the excel components that I am using do not have a setting whereby I can stop the system from generating empty reports! The end result, the system executes the job on schedule and is e-mailing blank reports to the end-user.

My workaround?
I have iterated files from tFileList into tFileProperties. I have connected the tFileProperties output into tMap so as to filter the ouput.
I am filtering based on tFileProperties filesize (the blank files are always the same size).
So two tMap outputs...one that states the FILTER clause, the other with NO clause stated...
What I am seeing is that ALL of my outputs, regardless of whether the filesize from tFileProperties qualifies in my filter expression, are going out on the tMap output that has NO filter expression defined??
MY ASSUMPTION:
I assume that if one output has a filter that qualifies/catches the data it would route it out on the tMap output (regardless of whether there are other outputs that do not make use of filter expressions)??
My filter expression is correct? row2.size.equals("121856")

Are there any other ways you could go about identifying a single line excel file (possibly a row count of the excel sheet = 1 type filter??)
Any help would be most appreciated as I'm scratching my head at the moment?
Thanks
TinTin
Employee

Re: tMap filter expression not working?

There seem to be two issues here.
For one thing, a filter expression on a tMap output applies only to that particular output. So if your "size" expression is true on the first output, the record should appear on that output, but it will also appear on the output that does not have a filter expression set. If you want your data only on one of the outputs, you need to use separate expressions on them: row2.size.equals(...) on the first one, and !row2.equals(...) on the second one. There is also a mode "Catch output reject" in the settings of an output. If that is set to true, then that output catches all records for which none of the filter expressions on the other outputs was true.
Second, it looks like your filter expression might be wrong. What exactly is the data type of the size column? Your expression would only work if it was a string column, but I doubt that. If it is an Integer column, then the correct expression should be row2.size == 123456.
Hope that helps.
Andre
One Star

Re: tMap filter expression not working?

Thank you for your time aspiegel!
Your reply has released me and my talend job is working like a charm!
YOU WROTE:
For one thing, a filter expression on a tMap output applies only to that particular output. So if your "size" expression is true on the first output, the record should appear on that output, but it will also appear on the output that does not have a filter expression set. If you want your data only on one of the outputs, you need to use separate expressions on them: row2.size.equals(...) on the first one, and !row2.equals(...) on the second one. There is also a mode "Catch output reject" in the settings of an output. If that is set to true, then that output catches all records for which none of the filter expressions on the other outputs was true.
-- I understand fully and made use of the Java operator != "not equal to" on the second output which has resulted in the job flowing through one or the other BUT NOT BOTH.
YOU WROTE:
Second, it looks like your filter expression might be wrong. What exactly is the data type of the size column? Your expression would only work if it was a string column, but I doubt that. If it is an Integer column, then the correct expression should be row2.size == 123456.
--Yip, was a datatype problem! I've used your expression and it's identifying the data as required.
Thank you for your inputs, they have truly helped me!
Kind Regards
TinTin
One Star

Re: tMap filter expression not working?

Hi all,
I saw your post on tMap component and filter of output reject.
I'm trying to implement a simple filter in my tMap component to filter null or empty values
I try to filter null values for a technical key :
- if the value is not null I insert the line in a fact table
- if the value is null I reject it and insert the line in a reject table
I want to use tMap filter and catch output reject ("row7.ID_Organisation==null") but I do not succeed so I use tFilterRow component (see screen1)
This works well but I thought it would be easier to maintain if implemented in the tMap component.
I think the problem is coming from the datatype (integer) of the ID_Organisation I want to filter.
Do you have any idea or suggestion that could help me solving this problem.
Thank you in advance