Write multiple tMap outputs to single component

One Star

Write multiple tMap outputs to single component

I'd like the tMap component to write multiple outputs to a single component.
Below is another post of mine which describes what I'm trying to achieve.
****************************** start of original post ***************************************
Hi folks,
I'm fairly new to TOS and am putting together my first 'real' job. I know what I'm trying to achieve but I'm not sure if I'm going about it the right way.
I have an input file which contains multiple record types. Different record type have different numbers of columns.
What I'm trying to do is to write out a single file with the columns of each record type moved such that each column in the output file has a single data type.
For example:
- input file may contain the following 3 records
My output file needs to be in the following format:
- column#1 should be string (is my record type)
- column#2 should be a date
- column#3 should be a string
- column#4 should be numeric.
Using my sample data from above my output file will be as follows:
I am using tMap to split the input file based on the record type (RT1, RT2 etc) and for testing have written those to tLogRow so I know I'm doing the correct processing.
How do I merge them back into one file ? I realise that I could write them all out to different files and then iterate through a filelist, but that seems like overkill to me. I'm typically processing less than 10 rows on each execution of the job.
I tried to use tUnite directly from the output of tMap but couldn't get it to accept multiple inputs. Seems strange but I could not get it to work. The first output from tMap dropped onto the tUnite component ok, but subsequent ones would not connect to it. The icon being shown was the "not allowed" one.
I then considered using a tBufferOutput from each row type of tMap, this is ok, but again trying to use tUnite to mwerge them back together I'm hitting the same problem.
What's the best way to achieve this ? All ideas and suggestions gratefully received.
****************************** end of original post ***************************************
Effectively I'm trying to use the tMap component to re-order the columns within a file based on a record type. There may be a better component to do this but I can't find one.

Re: Write multiple tMap outputs to single component

Have you tried to put all your outputs in different tbufferoutput ?
In a subjob you start with a tbufferinput and write it into a file, you should all line of all outputs in this file
One Star

Re: Write multiple tMap outputs to single component

I've actually worked around this by writing all outputs to different files and then merging them back again (which is pretty much what you suggested but without using tBuffers). I was trying to avoid doing that, hence my suggestion.
Can I do what I want to do ? yes.
But it would be easier with an enhancement to the product. That's what the suggestion is for.
One Star

Re: Write multiple tMap outputs to single component

Finally I did get rid of the Tbuffer component and wrote everything to two separate files early in the stream. Then used the Tjoin and it worked fine.
One Star

Re: Write multiple tMap outputs to single component

Thank you for sharing with us this insight is lacking. the attractiveness, usefulness, it feels when I finished reading the written language in this blog . Thanks to all authors. a bit of advice, however, that the author put forward his every interaction. - obat penghilang keloid yang sudah lama


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables


Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables


Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema