iteration on component execution

One Star

iteration on component execution

Hi everyboby,
I'm trying to create a new component that get an ArrayList of information. My schema is fixed (user can't change column). But with this code I get only the last set of value of my ArrayList...
Does anyone can help me?
In my Component_begin I've got this kind of code :
<%@ jet 
imports="
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.process.ElementParameterParser
java.util.List
java.util.Map
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
String username=ElementParameterParser.getValue(node, "__USERNAME__");
String password=ElementParameterParser.getValue(node, "__PASSWORD__");
%>
String user = <%=username%>;
String pass = <%=password%>;
int nb_line_<%=cid %> = 0;
//connect to Twitter
winterwell.jtwitter.Twitter twitter = new winterwell.jtwitter.Twitter(user,pass);
//get information from Twitter
java.util.List<winterwell.jtwitter.Twitter.User> userlist_<%=cid %> = new java.util.ArrayList<winterwell.jtwitter.Twitter.User>();
userlist_<%=cid%> = twitter.getFollowers();

<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
List<IMetadataColumn> columnList = metadata.getListColumns();
if (metadata!=null && conns.size()>0) {
for(IConnection conn:conns){
%>
for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
<%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
<%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
<%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
<%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
<%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
<%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
<%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
<%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
}

<%

}
}
}
%>
Highlighted
One Star

Re: iteration on component execution


You should put this part of your code in Component_main part
<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) {
IMetadataTable metadata = metadatas.get(0);
List<IMetadataColumn> columnList = metadata.getListColumns();
if (metadata!=null && conns.size()>0) {
for(IConnection conn:conns){
%>
for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
<%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
<%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
<%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
<%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
<%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
<%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
<%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
<%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
}

<%

}
}
}
%>
One Star

Re: iteration on component execution

I've split the code into two different parts by it doesn't work
begin :
<%@ jet 
imports="
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.process.ElementParameterParser
java.util.List
java.util.Map
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
String username=ElementParameterParser.getValue(node, "__USERNAME__");
String password=ElementParameterParser.getValue(node, "__PASSWORD__");
%>
String user = <%=username%>;
String pass = <%=password%>;
int nb_line_<%=cid %> = 0;
//connect to Twitter
winterwell.jtwitter.Twitter twitter = new winterwell.jtwitter.Twitter(user,pass);
//get information from Twitter
java.util.List<winterwell.jtwitter.Twitter.User> userlist_<%=cid %> = new java.util.ArrayList<winterwell.jtwitter.Twitter.User>();
userlist_<%=cid%> = twitter.getFollowers();

main :
<%@ jet 
imports="
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
org.talend.core.model.process.ElementParameterParser
java.util.List
java.util.Map
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
%>

<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
if (conns.size()>0) {
for(IConnection conn:conns){
if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)){

%>
for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
<%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
<%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
<%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
<%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
<%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
<%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
<%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
<%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
}

<%
}
}
}

%>
One Star

Re: iteration on component execution

I have a look to tRowGenerator and all is done in begin part. So forget what i said before.
What i think is :
your first loop should be :
for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
and then :
for(IConnection conn:conns){
Try it and tell me if it works
One Star

Re: iteration on component execution

No it doesn't work :'(
I had a look to most of components and I can't understand where the problem comes from.
Thanks a lot for your help, If you've got any other suggestions...
One Star

Re: iteration on component execution

If you've got any other suggestions... or THE solution.
When you look at tRowGenerator component, there is a trick : (see comments)
tTest_begin.javajet
<%@ jet 
imports="
org.talend.core.model.process.INode
org.talend.core.model.process.ElementParameterParser
org.talend.core.model.metadata.IMetadataTable
org.talend.core.model.metadata.IMetadataColumn
org.talend.core.model.process.IConnection
org.talend.core.model.process.IConnectionCategory
org.talend.core.model.metadata.types.JavaTypesManager
org.talend.core.model.metadata.types.JavaType
org.talend.designer.codegen.config.CodeGeneratorArgument
java.util.Map
java.util.List
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
String cid = node.getUniqueName();
%>
java.util.List<String> test = new java.util.ArrayList<String>(); // replace with your ArrayList from Twitter (I'm not register so i cannot test)
test.add("test1");
test.add("test2");
test.add("test3");
test.add("test4");
test.add("test5");
int nb_max_row_<%=cid%> = test.size();
<%
List<IMetadataTable> metadatas = node.getMetadataList();
if ((metadatas!=null)&&(metadatas.size()>0)) { // if1
IMetadataTable metadata = metadatas.get(0);
%>
for (int i<%=cid %> = 0; i<%=cid %> < nb_max_row_<%=cid%>; i<%=cid %>++) { // for2 row loop
<%
List< ? extends IConnection> conns = node.getOutgoingConnections();
for (int i = 0; i < conns.size(); i++) { // for3 connection loop
IConnection conn = conns.get(i);
if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)
&& (!conn.getLineStyle().hasConnectionCategory(IConnectionCategory.USE_HASH))) { // if2
for (IMetadataColumn column: metadata.getListColumns()) { // for4 metadata columns loop
%>
<%=conn.getName() %>.<%=column.getLabel() %> = test.get(i<%=cid %>);
<%
} // end for4
} // end if2
} // end for3
} // end for2
%>
// here there is no end for "if1"

tTest_end.javajet
<%@ jet 
imports="
org.talend.core.model.process.INode
org.talend.designer.codegen.config.CodeGeneratorArgument
"
%>
<%
CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
INode node = (INode)codeGenArgument.getArgument();
%>
} // end if1

What i get with a tFileOutputXML :
<?xml version="1.0" encoding="ISO-8859-15"?>
<root>
<row><test>test1</test></row>
<row><test>test2</test></row>
<row><test>test3</test></row>
<row><test>test4</test></row>
<row><test>test5</test></row>
</root>

In tTest_java.xml : there's just one parameter which is a schema type parameter
What i don't understand is why first "if" begins between <% %> and ends out of <% %> ? and why in end.javajet templates ? How does it really work ?
If there is a TalendTeam who can explain that ...
One Star

Re: iteration on component execution

Yes It works!!!!
Thank you very much.
I don't understand the } thing. I hope that someone from the Talend Team will explain us this fonctionnality.
I will publish my components soon
One Star

Re: iteration on component execution

Hi cahsohtoa,
How did this component development go ?
Did you finally manage to create a twitter component ?
I am quite interested in your work, please tell me if I can give you a hand in something.
Best regards,
Maxime.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download