How to handle looping with in aloop in tJavaFlex?

One Star

How to handle looping with in aloop in tJavaFlex?

I have tJavaFlex code its like
In Main code: 
Code for Process all input records until EOF.
Used while condition.
In Main code:
for loop to porcess every 1000 record chunk.
In End code:
close the Main code brace.
In main code 'row2.recordOut=i' is getting replaced for every iteration, and giving invalid target file. 
How to do this recursion technique in talend using tJavaFlex, without replacing the output records?
Four Stars

Re: How to handle looping with in aloop in tJavaFlex?

Hi, 
"Start code", "main code" and "end code" refers to the states / stages of the tJavaFlex component. "Start code" and "end code" are executed on initiation and termination while the "main code" is executed for every set of data. 
If I understand you correctly, your row2.recordOut refers to a file and you want to read multiple files. Is this correct? In this case you should place the full code for reading the whole file into the "main code" (while perhaps initiating the variables in the "start code"). 
Could you please add a screenshot and explain what you're trying to do in more detail, if I misunderstood your post? 
Best Regards
Philipp Borst
One Star

Re: How to handle looping with in aloop in tJavaFlex?

Thanks Philipp for your reply.  
These are the steps I did in Job: 
tLibraryLoad(externalJar) --> tFileInputRegEx(Column:RecordInput--->tMap(Columns:ID,RecordInput)-->tJavaFlex-->tFileOutputDelimited 
In tMap, created ID column to input records using expression-Numeric.sequence("s1",1,1) to prepare unique sequence ID.
Following is the tJavaFlex code:
StartCode:
List<String> list=new ArrayList<String>();
int chunkSize = 100000; 
MainCode:
if(out1.ID%chunkSize== 0) //out1 is link name, ID(Sequence ID) is the column name in tMap,
{
   ArrayList<String> listData =ExternalJarOperations(list);
   for(String s:listData)
  {
    System.out.println("Out record is:"+s);//Checking processed record
    row2.LogRowOut=s; //Putting the processed record to target file
  }
  list.clear();
  }else{
  list.add(out1.LogRowIn);
}
End Code:
System.out.println("Transformation completed");
----------------------------------------------------------------
Here 
 System.out.println("Out record is:"+s); gives processed records in console correctly. 
but row2.LogRowOut=s; is replacing every record in target and getting only last record in target.
How to could we solve this?
Four Stars

Re: How to handle looping with in aloop in tJavaFlex?

Thanks for the elaboration. So you basically want to create multiple output rows (for each iteration of the loop) in the tJavaFlex for one input-row? 
I don't know any way to do this directly with a single tJavaFlex-component. However since we have a lot of tools at our disposal, there are some possible workarounds.
One way I've used in the past is desscribed in this stackoverflow example , but I don't think that this will fit your needs.
I recommend that you work with the globalMap to save the entries. Just write the ArrayList to the globalMap, then spawn the rows with a tRowGenerator or a tFixedFlowInput combined with a loop.
Does this workaround solve your problem?
One Star

Re: How to handle looping with in aloop in tJavaFlex?

You are saying that store all records to an arraylist, then add this arraylist to GlobalMap (key,<AL>)?.
It's a solution but performance issues will occur for big file. I worked around this previously for 900Mb file on 6GB ram, its got failed with heap issue, even increasing the JVM heap memory from talend settings.
Four Stars

Re: How to handle looping with in aloop in tJavaFlex?

It doesn't matter how you save the "results" that you're currently writing to row2.LogRowOut, but I'm afraid that there's no way to create a new row for each resultset in your ArrayList listData within a tJavaFlex. You could save the data in the GlobalMap, a context variable, a file, the database, .... until you've read the whole file and you're ready to process the data.
Perhaps someone else knows a method to create new rows within a tJavaFlex component. You could also do this by building a new and custom component instead of using the tJavaFlex.
One Star

Re: How to handle looping with in aloop in tJavaFlex?

Thanks Phipp for your reply.
Yes, I used Global Map concept. 
tFileInput--->tJavaFlex(get all rows from input and store them in AL, in end code save AL to globalMap)-->tJavaFlex(Get AL from GlobalMap and process list with externalJar, In end code put outputdata to output file--->output file.
Here we have a problem that GlobalMap depends on JVM heap(RAM). System Hung for huge file because we are stored all file data into GlobalMap.
What would be the solution for this? I'm very new to Talend so please give me the idea briefly!
Thanks.