Using Same data to join in tmap

Four Stars

Using Same data to join in tmap

Hi,

 

I have a data source (Oracle Table) which I want to use to multiple times in my tMap. Since I have two different keys to join with the source that, is there any way I can use the same data flow twice? 

 

I tried tReplicate, but it will not allow me to join the main flow twice to same tMap.

I tried to use a tMap after the table unload with two outputs and connect it to the tMap.

None of the solution looks to work. 

 

Since this is a Dev environment the data volume in the below picture is less, but in Production it will be in Millions. Any assistance appreciated. 

 

PS: I am new to Talend.

 

Multilookup.JPG


Accepted Solutions
Thirteen Stars

Re: Using Same data to join in tmap

one of the possible solutions - use tHashInput / tHashOutput components, store once, use many

but it could use a lot of memory for big datasets.

 

other possible solutions, what could work better than database request - store results into local csv file, and re-use it when necessary.

 

third (depend from job logic) - relocate lookups on database side.

 

depending from design - each of this 3 ways could be the best choice.

 

-----------

All Replies
Thirteen Stars

Re: Using Same data to join in tmap

one of the possible solutions - use tHashInput / tHashOutput components, store once, use many

but it could use a lot of memory for big datasets.

 

other possible solutions, what could work better than database request - store results into local csv file, and re-use it when necessary.

 

third (depend from job logic) - relocate lookups on database side.

 

depending from design - each of this 3 ways could be the best choice.

 

-----------

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.