Five Stars

Loop for job and output data usage in expression fields.

Hello,

I need to collect data from api.But its have row limit.I want to mean i can't collect all data in single operation.Its have pages and every page have row limit.I need to loop this job (totalpagecount) times for collect all data.

 

TWebservice > TNormalize > TMap > TFileOutputExcel 

 

1. image : Current page value for input

2. image : Current page and total page values for output

Requirements:

My job need to run (Total Page Count) times.

currentpage value need to  increase in every one loop

Loop need to be stop ^^

 

Example:

Input currentpage field will be start from 0.So its need to increase in every run.

Loop need to stop when currentpage = totalpagecount.

 

Loop run up to currentpage = totalpage

{

TWebservice > TNormalize > TMap > TFileOutputExcel

^<Tbufferoutput < (currentpage value???? with increasing every one loop)

}

 

 How can i do it ?I hope for solution.

  

One more question : 

how can i use mysql queries in expression fields ?I check this too much times but i can't found any topic.

 

Looking for help.

Thanks Smiley Happy

2 ACCEPTED SOLUTIONS

Accepted Solutions
Community Manager

Re: Loop for job and output data usage in expression fields.

Hi
Just realize that twebservice don't accept an iterate link, need to redesign the job as below:
main job:
twebserice1--main--tJavaRow
|
onsubjobok
|
tLoop1--iterate--trunJob

trunJob: call the child job and pass the current loop value to child job. To do it, select the context variable which is defined in child job from the Context param table on tRunjob, and set its value as:
((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))

child job:
tWebservice2--main--tLogRow

Define a context variable let's called it current_value with int type in the Context viewer.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Community Manager

Re: Loop for job and output data usage in expression fields.

What's the error message? To assign the page value to a context variable or global variable like this:
context.page_value=input_row.columnName;
//context variable page_value is defined as int type.
Then, you can use this variable as To field: context.page_value

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
9 REPLIES
Community Manager

Re: Loop for job and output data usage in expression fields.

Hi
First of all, get the value of total page and store it to a context variable, this variable will be the To value on tLoop component which is used to loop each page, eg:
twebserice1--main--tJavaRow
|
onsubjobok
|
tLoop1--iterate--tWebservice2--main....

twebservice1: get the value of total page if this value comes from a webservice.

twebservice2: pass the current page as input parameter with this expression:
((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))

Why do you ask 'use mysql queries'? What value do you need to execute a query to get?

Regards
Shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: Loop for job and output data usage in expression fields.

Hello,

Thanks for this but i can't itirete tloop to twebservice.
Mysql queries for other jobs.Not important for now.
Community Manager

Re: Loop for job and output data usage in expression fields.

Hi burakbb
What's the issue? Why you said you can't iterate tloop to twebservice? Maybe I don't understand your problem well?
----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: Loop for job and output data usage in expression fields.

Hello,
Sorry i want to mean tloop can't be itirate to twebservice.
I am using open talend studio data integration.Could it be related to this?

Edit:

When i try to itirate them, its not complete.I can't use itirate feature on tloop to twebservice.Because its not allowed.Open talend studio data integration not allow this.

 

Community Manager

Re: Loop for job and output data usage in expression fields.

Hi
Just realize that twebservice don't accept an iterate link, need to redesign the job as below:
main job:
twebserice1--main--tJavaRow
|
onsubjobok
|
tLoop1--iterate--trunJob

trunJob: call the child job and pass the current loop value to child job. To do it, select the context variable which is defined in child job from the Context param table on tRunjob, and set its value as:
((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"))

child job:
tWebservice2--main--tLogRow

Define a context variable let's called it current_value with int type in the Context viewer.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: Loop for job and output data usage in expression fields.

Hello Shong,

Ty so much.Already i fix that but its more better.I will apply your design.(I used uneffected component(Trest) for itirete them ^^)

But i am still not done.I need to put webservice_1 total page value to tloop_1 "To" field.I tried somethings but every time i got error.I can't set global variable in tjavarow.

Can you help me for this ?

Community Manager

Re: Loop for job and output data usage in expression fields.

What's the error message? To assign the page value to a context variable or global variable like this:
context.page_value=input_row.columnName;
//context variable page_value is defined as int type.
Then, you can use this variable as To field: context.page_value

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Five Stars

Re: Loop for job and output data usage in expression fields.

Thank you so much.Its done Smiley Happy Everything working correctly.

 

Community Manager

Re: Loop for job and output data usage in expression fields.

Great, thanks for your feedback!
----------------------------------------------------------
Talend | Data Agility for Modern Business