[resolved] how to merge two files and get one output

One Star

[resolved] how to merge two files and get one output

i have this sinple job of transfering a data from csv to db table.
all i want is to check the rejected records and merge them into one file.
the tfileinputdelimited will reject some data due to schema validation.
and the touputmssql will also reject some records due to constraints.
all i want is to merge the rejected records in one single file.
i tried tfilelist-didnt work,
tcreatetemporaryfile too..didnt got the expected result.
help me in this matter, plz find the attached screenshot of my job for better undrstanding of the problem
thnks in advance

Accepted Solutions
Community Manager

Re: [resolved] how to merge two files and get one output

Hi
You can use tHashOutput component to cache the record in memory, instead of file with tFileOutputDelimited component, and use tHashInput to read the data from memory back again in the next subjob, use a tUnite to merge the two data flow and output the result to a file in the next subjob.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

All Replies
One Star

Re: [resolved] how to merge two files and get one output

hi- hey even i have the same doubt.
waiting for experts to reply-------
One Star

Re: [resolved] how to merge two files and get one output

You can use outputdelimited and recall this file in the same job.
Community Manager

Re: [resolved] how to merge two files and get one output

Hi
Please take a look at this KB article, it explains the problem and provides solutions.
https://community.talend.com/t5/Design-and-Development/Can-I-create-a-Job-with-multiple-paths-from-a...
Shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] how to merge two files and get one output

hi shong-
thanks for reply-
but what i want to do is merge records ie. row.
and tmap will work on columns.
i have two csv files being created in the same job will same schema.
now suppose file1.csv has 10 records, and file2.csv has 5 records.
then i want to have a file3.csv with 15 records as the ouput(is addition of file1 records and file2 records)
note: i dnt want the file1.csv and file2.csv to be saved anywhere.
Community Manager

Re: [resolved] how to merge two files and get one output

Hi
You can use tHashOutput component to cache the record in memory, instead of file with tFileOutputDelimited component, and use tHashInput to read the data from memory back again in the next subjob, use a tUnite to merge the two data flow and output the result to a file in the next subjob.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] how to merge two files and get one output

hi shong-
i m unable to find/search for the component "tHashoutput".
i guess it is because i am using a freely available version of talend studio.- m i correct??
can u help with with an altrnative solution of merging two files??
thnkx in advance..
Community Manager

Re: [resolved] how to merge two files and get one output

Hi
The tHashInput/tHashOutput are also available in community version, they might have been hidden, go to the Project settings, then Designer, Palette, move them from the Hide side to Show side.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] how to merge two files and get one output

thank u so much shong--
this really helped me--
Regards,
Roni V.
Six Stars

Re: [resolved] how to merge two files and get one output

 

 

please let me know how did u do it..? I am facing the same issue. 

 

5.pngHi, Can you

Moderator

Re: [resolved] how to merge two files and get one output

Hello,

A "Circle" in work flow is not allowed in talend.
Could you please have a look at document about:https://community.talend.com/t5/Design-and-Development/Can-I-create-a-Job-with-multiple-paths-from-a...

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Ten Stars

Re: [resolved] how to merge two files and get one output

In the tHashOutput_2 settings, check the "Link with a tHashOutput" box and choose tHashOutput_1 from the dropdown.

Start a new subjob with a tHashInput, link the tHashInput to tHashOutput_1 in the same manner as above. Specify the same schema in tHashInput that you used in your tHashOutput components. Connect the tHashInput to the output component of your choice.

Link the subjobs by creating an OnSubjobOk link between tFileInputDelimited_1 and tHashInput_1