Four Stars

Process multiple Files with different schemas

Hello all,

maybe this issue has shown up before but for me i didnt find the right answer.So here we go. I try to explain as simple as possible what iam looking for. And for sure this is no rocket science Smiley Happy

Ok, I have different incomming Datafiles, some are .xls some are .xlsx and some .csv oder edifact or .xml etc. etc.

They all have different schemas.

Now i want to itterate over this "heap" of Files.

Each time I start processing a single File i want to "look" into a DB Table or .txt file or where ever i can store  information whith wich Schema this particular FIles has to be processed.

As an Identifier i want to use the Filename or maybe the EmailSender if this has come by email. Any suggestions?


Thank u all!!


Four Stars

Re: Process multiple Files with different schemas

PS: offcourse i will use different tfileinputs for .csv and .xml and .xlsx

Re: Process multiple Files with different schemas


Maybe you can load your multiple Files with different schemas into tables firstly and then extracting data from DB tables through t<DB>Input component.

Here is a dynamic schema feature in talend subscription version which allows you to design schema with an unknown column structure (unknown name and number of columns).

Best regards



Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.