tMap Lookup - Reload at each row - behind data flow -huge Lookup Table

One Star

tMap Lookup - Reload at each row - behind data flow -huge Lookup Table

Hi
I work with TIS 3.2.3.
My problem is to look up values for a few data rows in a Lookup Table with well above 1 Mio rows. Obviously the loading of the whole lookup table is very expensive.
So I figured out a dynamic statement which I combined with the Reload at each row option. But now comes the problem. The lookup process is behind the row - process. If you look at the option label "Reload at each row" I must ask: Is it "Reload before each row" or "Reload after each row".
My Job Set up (See picture) was set up with a java component to write the row into globalMap and then the Oracle Lookup Statement was constructed with that globalMap Value. Unfortunately the whole process is not really in sync The trace shows always the previous value from the row before in the lookup branch.
Has anybody another idea to get quick lookup for a few values from a huge table?
Thank you
Benjamin
One Star

Re: tMap Lookup - Reload at each row - behind data flow -huge Lookup Table

Hi
I did work a little bit on the job, replaced some components and suddendly the oracle component did work properly with its dynamic statement. Maybe it comes from using context instead of globalVar? I don't know. It seems, that the other issue "tMap with dynamic sql lookup" had the same basic question and solved it similarly.
Thanks anyway

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog