How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

Seven Stars

How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

Suppose tFileList_1 fetches the file Sub_20180522.xlsx, then tFileList_2 must  read this Pay_20180522.xlsx file the suffix should be same of both files similarly in tFileList_3 and tFileList_4 

 

how can i implement this kind of Structure in Talend Data Integration?? 


Accepted Solutions
Thirteen Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@meetmahajan,can you try with below way as a pattern of tFileList.

 

"Sub_"+TalendDate.getDate("yyyyMMdd")+".xlsx"
"Pay_"+TalendDate.getDate("yyyyMMdd")+".xlsx"

Manohar B

All Replies
Thirteen Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@meetmahajan,can you explain detailed,what eactly are you looking for?

Manohar B
Thirteen Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@meetmahajan,can you try with below way as a pattern of tFileList.

 

"Sub_"+TalendDate.getDate("yyyyMMdd")+".xlsx"
"Pay_"+TalendDate.getDate("yyyyMMdd")+".xlsx"

Manohar B
Seven Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@manodwhb Suppose one folder consist of  following files:

 

Pay_20180522.xlsx

Pay_20180523.xlsx

Pay_20180524.xlsx

Sub_20180522.xlsx

Sub_20180523.xlsx

Sub_20180524.xlsx

 

If tFileList_1 executes Pay_20180522.xlsx then compulsorily tFileList_2 should execute Sub_20180522.xlsx  and store this data into external file or database 

same iterating again the if tFileList_1 executes Pay_20180523.xlsx then tFileList_2 must Execute the file of Sub with same date as Pay_20180523.xlsx means it should execute Sub_20180523.xlsx

Seven Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@manodwhb

 

"Sub_"+TalendDate.getDate("yyyyMMdd")+".xlsx"
"Pay_"+TalendDate.getDate("yyyyMMdd")+".xlsx"

This will only Fetch files of current date !! that is the issue , i want to Fetch thousands of files of previous year, which having same format as 

 

Sub_YYYYMMDD.xlsx

Pay_YYYYMMDD.xlsx

Thirteen Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@meetmahajan,in a tFileLists,if you specify Order by>>By file name and Order action>> ASC in Basic setting.

you will get the same date file accross the filelists.

and use the pattern like below.

 

"Sub_*.xlsx"
"Pay_*.xlsx"

 

Manohar B
Seven Stars

Re: How can i simultaneously fetch multiple files from same folder with same suffix, if tFileList1 reads "Sub_20180522.xlsx" then tFileList2 must read "Pay_20180522.xlsx" using Talend Open Studio

@manodwhb

i Executed it already, what happens when i execute this type of statements

"Sub_*.xlsx"
"Pay_*.xlsx"

 

Sub_20180522 maps with Pay_20180523 which is inconvenient and data will corrupt, I want to create a job that execute following points 

Suppose output file is May_2018

 

  • Sub_20180522 should be mapped with only Pay_20180522, not other files and store output in May_2018.
  • Similarly from 20180501 to 20180530 each file of Pay should be mapped with an exact same date as Sub and Store it in May_2018
  • each files contains ID and Pay in Pay
  • each files of Sub contains ID and Name
  • May_2018 contains, ID,NAme and Pay.
  • each ID and Name will be the same but Pay will different.
  • with the use of this "Sub_*.xlsx" & "Pay_*.xlsx", my data is corrupted due to mismatching.