Collect a fixed amount of rows

Highlighted
Six Stars

Collect a fixed amount of rows

hi guys, i have a question again:

I have a certain number of rows and would like to put them into packages of the same size. Let's say I have 500 rows and always want to collect and summarize 20.

 

For example:

Row1

Row2

Row3

...

Row21

Row22

...

 

Should be:

Row1|Row2|Row3|...

Row21|Row22|... and so on

 

I've been looking for a solution for several hours now, but I haven't found any. I hope you can help me.

 

Best regards,

BooWseR

 


Accepted Solutions
Twelve Stars

Re: Collect a fixed amount of rows

hi, you have 2 questions.
first is to group by n rows.
you can use tmap creating seqence and using modulo to get a unique index by n lines Numeric.sequence("seq1",0,1)%n+1 -- id
then use taggregate to group lines in list.

an other solution is to manualy concaténate fields into tmap variable
var.var1 = var.var1 + row1.value
then use taggregate to keep only the last line.

good luck

Francois Denis

Tag as "solved" for others! Kudos to thanks!


All Replies
Twelve Stars

Re: Collect a fixed amount of rows

hi, you have 2 questions.
first is to group by n rows.
you can use tmap creating seqence and using modulo to get a unique index by n lines Numeric.sequence("seq1",0,1)%n+1 -- id
then use taggregate to group lines in list.

an other solution is to manualy concaténate fields into tmap variable
var.var1 = var.var1 + row1.value
then use taggregate to keep only the last line.

good luck

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Six Stars

Re: Collect a fixed amount of rows

Hi Francois, thank you for your answer.

 

In the meantime, I have been able to find my own solution, which also works quite well. Let me show you:

 

image.png

 

First I get the data from a REST interface. Then I extract the data I need from the JSON, split it with "tNormalize" and write it into an ArryList with tJavaFlex.

Then I iterate with tLoop in predefined steps (10 each) up to the maximum of the ArrayList.

 

image.png

 

I'm pretty sure I could have used the output from the JSON as a list somehow, but I couldn't get it right. I'm sure that your solution will certainly perform better, but currently mine works. When I find the time, I'll try your solution. Anyway, thanks for your help.

 

Best regards,

BooWseR

Twelve Stars

Re: Collect a fixed amount of rows

hi mark your topic as solved,
just think about maintenance, less java code easier is the understanding of a talend job.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Seven Stars

Re: Collect a fixed amount of rows

Hi !

 

Wow, so much brain functioning for a simple thing ^^

Use tSampleRow and you should be ok Smiley Happy

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now