One Star

reuse look up flow data

it will be faster if a look up data is reused when it's lodaed twice or more in a job.
2 REPLIES
Community Manager

Re: reuse look up flow data

Hi
You can store lookup data into memory using tHashOutput component, and then reuse the lookup from memory, it will be faster than loading the lookup each time from input source. For example:
tMysqlInput--main--tHashOutput
|
onsubjobok
|
tFileInputDelimain---main--tMap---tLogRow
|
lookup
|
tHashInput(link to the tHashOutput)
Load model: Reload at each row.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: reuse look up flow data

Hi Shong
I have a doubt here. Isnt the reload at each row goes slow as number of record increases in main file.
I mean, If the main file has 1 milion records and look 1 milion.
Please correct me if am wrong