set priority to read lookup files

One Star

set priority to read lookup files

Hi,
 
We have requirement to update main table (>30millions) based on business rules & lookup files. There are 46 lookup files with different schema & business rule. The priority to update is set for each lookup file & it need to done accordingly.
 
File PriorityFile Name
1AA.txt
3BB.txt
2XX.txt
4ZZ.txt
 
In this case the update will happen in order: AA,XX,BB,ZZ

The solution which we can think is:

Main Table -->(Lookup AA.txt)-->tMap1--->(Lookup XX.txt)-->tMap2--->(Lookup BB.txt)-->tMap3--->(Lookup ZZ.txt)-->tMap4--->so on...
Is this approach feasible? Whether it is achievable with one tMap?

 
Please suggest optimum development approach for this scenario.
Thanks
One Star

Re: set priority to read lookup files

Whether it's possible in Talend to set priority for lookup files?
Forteen Stars

Re: set priority to read lookup files

read file(tFileInputDelimited) -> sort file (tSortRows) -> iterate over sorted rows (tFlowToIterate)
-----------

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now