One Star

set priority to read lookup files

Hi,
 
We have requirement to update main table (>30millions) based on business rules & lookup files. There are 46 lookup files with different schema & business rule. The priority to update is set for each lookup file & it need to done accordingly.
 
File PriorityFile Name
1AA.txt
3BB.txt
2XX.txt
4ZZ.txt
 
In this case the update will happen in order: AA,XX,BB,ZZ

The solution which we can think is:

Main Table -->(Lookup AA.txt)-->tMap1--->(Lookup XX.txt)-->tMap2--->(Lookup BB.txt)-->tMap3--->(Lookup ZZ.txt)-->tMap4--->so on...
Is this approach feasible? Whether it is achievable with one tMap?

 
Please suggest optimum development approach for this scenario.
Thanks
2 REPLIES
One Star

Re: set priority to read lookup files

Whether it's possible in Talend to set priority for lookup files?
Twelve Stars

Re: set priority to read lookup files

read file(tFileInputDelimited) -> sort file (tSortRows) -> iterate over sorted rows (tFlowToIterate)
-----------