One Star

[resolved] Can't filter out Rows

Hi!
I'm trying to create a new component to automatical filter out all empty rows.
This component also serves as a learning experience for me so it's my first own component.
My Goal is to have a component which i can just drop into the flow to filter out the empty rows.
I know that this can be achieved with tFilterRow but i want a component which does it without any extra work.
At the moment the flow must have only variables of type String. I will implement the detection of the type later when the filtering works correctly.
 
My problem is that if i don't write the content of input_row to the output_row, Talend automatically writes the values from the row before into the output_row instead.
I got it working by putting a continue into the if-statement but this only works correct for flows that are just a single line and surely isn't the right way to do it.

I tried to write the rejected rows into a second output instead but it didn't work. This just creates an empty output with the same rowcount as the input and doesn't change the filter output at all
Am i missing something? I read on a tutorial from powerupbi that i need DATA_AUTO_REPLICATE="false" in my headers but if i put it there the component can't be loaded and the Talendlog says, that DATA_AUTO_REPLICATE isn't allowed in the headers
I do have HAS_CONDITIONAL_OUTPUTS="true" in my Headers
I'm using Talend Data Fabric 6.2.1

Regards,
Nils Preuschoff
tFilterEmptyRows_begin.javajet
<%@ jet
imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.metadata.types.JavaType
java.util.List
java.util.Map
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
%>

int nb_line_<%=cid%> = 0;
int nb_line_empty_<%=cid%> = 0;
int nb_line_ok_<%=cid%> = 0;

tFilterEmptyRows_main.javajet
<%@ jet
imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.metadata.types.JavaType
java.util.List
java.util.Map
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
%>
boolean empty = true; <%
List< ? extends IConnection> incoming = node.getIncomingConnections();
if ((incoming != null) && (incoming.size() > 0)) {//A
IConnection income2 = incoming.get(0);
IMetadataTable income = income2.getMetadataTable();
if (income != null) {//B
List<IMetadataColumn> columns = income.getListColumns();
for( IMetadataColumn col : income.getListColumns() ) { %>


if (<%=income2.getName()%>.<%= col%> != null && !(<%=income2.getName()%>.<%= col%>.trim().isEmpty())) { //D

empty = false;
}//D
<%
}%>

if(empty) {//E
nb_line_empty_<%=cid %>++;
continue;

} else {
<%

List< ? extends IConnection> filter_conns = node.getOutgoingConnections("FILTER");
if ((filter_conns != null) && (filter_conns.size() > 0)) {//F
IConnection metadata = filter_conns.get(0);%>
<%=metadata.getName()%> = new <%= metadata.getName() %>Struct();
<%
for(IMetadataColumn col : income.getListColumns() ) {//G
%>

<%=metadata.getName()%>.<%=col %> = <%= income2.getName() %>.<%= col %>;

<%
}//G
}//F
%>
nb_line_ok_<%=cid%>++;
}//E
<%
}//B
}//A
%>

nb_line_<%=cid %>++;
globalMap.put("<%=cid%>_NB_LINE",nb_line_<%=cid%>);
globalMap.put("<%=cid%>_NB_LINE_EMPTY",nb_line_empty_<%=cid%>);
globalMap.put("<%=cid%>_NB_LINE_OK",nb_line_ok_<%=cid%>);


tFilterEmptyRows_java.xml
<COMPONENT>
<HEADER
PLATEFORM="ALL"
SERIAL=""
VERSION="0.1"
STATUS="ALPHA"

COMPATIBILITY="ALL"
AUTHOR="Nils Preuschoff"
RELEASE_DATE="20160829A"
SCHEMA_AUTO_PROPAGATE="true"
HAS_CONDITIONAL_OUTPUTS="true"
DATA_AUTO_PROPAGATE="false"
STARTABLE="false"
PARTITIONING="AUTO"
LOG4J_ENABLED="true"
>
<SIGNATURE/>
</HEADER>

<FAMILIES>
<FAMILY>Quinscape</FAMILY>
</FAMILIES>

<DOCUMENTATION>
<URL/>
</DOCUMENTATION>

<CONNECTORS>
<CONNECTOR CTYPE="FLOW" MAX_INPUT="1" MAX_OUTPUT="0"/>
<CONNECTOR NAME="FILTER" MAX_OUTPUT="1" CTYPE="FLOW" BASE_SCHEMA="FLOW" COLOR="086438" />
<CONNECTOR CTYPE="SUBJOB_OK" MAX_INPUT="1" />
<CONNECTOR CTYPE="SUBJOB_ERROR" MAX_INPUT="1" />
<CONNECTOR CTYPE="COMPONENT_OK" />
<CONNECTOR CTYPE="COMPONENT_ERROR" />
<CONNECTOR CTYPE="RUN_IF" />
</CONNECTORS>

<PARAMETERS>
<PARAMETER NAME="SCHEMA_FILTER" FIELD="SCHEMA_TYPE" CONTEXT="FILTER" NUM_ROW="1" READONLY="true"/>

</PARAMETERS>

<CODEGENERATION/>

<RETURNS>
<RETURN NAME="NB_LINE" TYPE="id_Integer" AVAILABILITY="AFTER"/>
<RETURN NAME="NB_LINE_OK" TYPE="id_Integer" AVAILABILITY="AFTER"/>
<RETURN NAME="NB_LINE_EMPTY" TYPE="id_Integer" AVAILABILITY="AFTER"/>
</RETURNS>

</COMPONENT>
1 ACCEPTED SOLUTION

Accepted Solutions
One Star

Re: [resolved] Can't filter out Rows

After i had some time again to look at this i finally found a solution
I just needed to set the outputrow to null if the input is empty
4 REPLIES
Moderator

Re: [resolved] Can't filter out Rows

Hi,
Have you already checked article about:TalendHelpCenter:How to create a custom component?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: [resolved] Can't filter out Rows

Hi Sabrina,
sorry for the late response here.
Yes i did read it and also this one
I still can't find my problem with this.
I can add the content of the other files, too if that helps
Best regards
Nils
One Star

Re: [resolved] Can't filter out Rows

After i had some time again to look at this i finally found a solution
I just needed to set the outputrow to null if the input is empty
Moderator

Re: [resolved] Can't filter out Rows

Hi NPreuschoff,
Thanks for posting that you have resolved this issue by yourself and sharing your solution here.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.