One Star

process multiple files in Talend

can someone tell how to process multiple files in Talend and store the output in a oracle table??
I tried using tfileList component but i was unable to insert data into table.
My file extension is also ".unl" which no file related component is reading data..
So could someone help me with this??
11 REPLIES
Moderator

Re: process multiple files in Talend

Hi,
Can you convert your .unl files to flat files(excel) and then use tfilelist to achieve your goals?
The work flow looks like:tfilelist-->tfileinputexcel-->output
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Twelve Stars

Re: process multiple files in Talend

most important not extension, but what inside
open it with any flat text editor - notepad++, textedit and etc, than You (and we are) will have idea - what You try to read?
-----------
One Star

Re: process multiple files in Talend

Hi,
Thanks for the replies.
I should not change the file format as it is production environment.What ever the .unl files I get in the directory differed by a sequence number I should read them and process them and store the required fields in the oracle table as output.
The data in the .unl file is pipe delimited (for example : 12345|active||P|hello) like this.
But when I iterate the files in the directory its reading only first file and not reading other files.
The sample file names in the soure directory will be like
sample20161115_SOMECODE1_SEQNUMBER.unl
sample20161115_SOMECODE2_SEQNUMBER.unl,etc
So how can i iterate??I mean what filemask should i keep in tfilelist component??
Any help would be greatful
Twelve Stars

Re: process multiple files in Talend

of course - tFileList (use as pattern "*.unl") -> tFileInputDelimited -> tMap ->output
all must work as switzerland watches 
-----------
One Star

Re: process multiple files in Talend

Hi,
Thanx for the reply.
But still my problem was not resolved.
I have three files in a directory
mon20161113_103_941462.unl
mon20161113_103_941463.unl
mon20161113_103_941464.unl
But when I gave filemask as "mon20161113_103_*.unl" in tfilelist and in tfileinputdelimited I created metadata with mon20161113_103_941462.unl file.
But when i run my job its showing 3 execs finished but when i see output the first file data is only stored 3 times but not 2 and 3 files.
Please help me with this.
Twelve Stars

Re: process multiple files in Talend

it mean - You have error somewhere also ;-)
we of course can try to guess, but will be much better if You show Your Job ... much more easy
without information, just trust me - it work! Smiley Very Happy even with hundreds of files, sure simple error
-----------
One Star

Re: process multiple files in Talend

Attached is the job...pls help me with this
Moderator

Re: process multiple files in Talend

Hi,
We can't see the screenshot on our side. Could you attach it on the forum, please? That would be great.
Screenshots works only if you drag&drop the image directly in the editor window. 
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: process multiple files in Talend

PFA
One Star

Re: process multiple files in Talend

HI,
Thanks.Faced some issue while uploading
Moderator

Re: process multiple files in Talend

Hi,
Can you click "UPLOAD" button to attach your screenshot?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.