set priority to read lookup files

One Star

set priority to read lookup files

We have requirement to update main table (>30millions) based on business rules & lookup files. There are 46 lookup files with different schema & business rule. The priority to update is set for each lookup file & it need to done accordingly.
File PriorityFile Name
In this case the update will happen in order: AA,XX,BB,ZZ

The solution which we can think is:

Main Table -->(Lookup AA.txt)-->tMap1--->(Lookup XX.txt)-->tMap2--->(Lookup BB.txt)-->tMap3--->(Lookup ZZ.txt)-->tMap4--->so on...
Is this approach feasible? Whether it is achievable with one tMap?

Please suggest optimum development approach for this scenario.
One Star

Re: set priority to read lookup files

Whether it's possible in Talend to set priority for lookup files?
Thirteen Stars

Re: set priority to read lookup files

read file(tFileInputDelimited) -> sort file (tSortRows) -> iterate over sorted rows (tFlowToIterate)

Cloud Free Trial

Try Talend Cloud free for 30 days.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.