Five Stars

Multiple File Processing

Hi All,

I have a requirement where i need to process multiple files from a folder using some specific pattern , but we cant use tfilelist component as its not available in  big data platform. can anyone suggest a solution to how can i process.

I tried with context variable in tfileinputdelimeter component but its not processing.

 

for ex: the context which i used is "context.varname + "<partoffilename>*.dat"

Am using talend Big Data Platform 6.3.1. and we are designing jobs in big data batch. and the files are in Azure.

 

Please anyone can suggest some solution

 

Thanks in advance,

Lmit

2 REPLIES
Six Stars

Re: Multiple File Processing

Hi Lmit,

We can write a java routine where we can get the list of filenames.

you may try the below desgin

Tfixedflowinput-->tjavrow-->tflowtoiterate-->tfileinputdelimeted--->Target

 

Thanks

Jilani syed

Five Stars

Re: Multiple File Processing

Hi Jilani syed,

 

thanks for your quick reply, but i cant implement your logic as tflowtoiterate is not there inmy version

 

Thanks,

Lmit