One Star

One output as multiuple inputs?

Hi,
Is there anyway to use the output of one component as the input to multiple componets?
Thanks,
Jay
10 REPLIES
One Star

Re: One output as multiuple inputs?

why not use a tMap with one input and multiple outputs to multiple components?
One Star

Re: One output as multiuple inputs?

Is that the most efficient way to do it? Does that not create mutiple copies of the same dataset in memory?
One Star

Re: One output as multiuple inputs?

hi,
i don't know if it works, but i can design it.
One Star

Re: One output as multiuple inputs?

Thanks Chris.
It seems that it works for data inputs but not for the other compnents. Try going from tMap or another porcessing node for example.
One Star

Re: One output as multiuple inputs?

Any chance of this coming in the next version Talend?
Employee

Re: One output as multiuple inputs?

In a next version of TOS, we will have only one component with multiple outputs: tReplicate.
tMap will also be able to output several distinct rows (but tMap is a very special component).
One Star

Re: One output as multiuple inputs?

Hi,
I am trying to use this in my value replacement issue. I need to count at several levels and then put it all together.
How should I do this?
The only conents that the flow into more then one component are the input ones...
I was also trying to feed a tMap and tAggregateRow into a tmap to use the counts but I couldn't do this either...
Any suggestions would be greatly appreciated!
Many Thanks,
Jay
881
What I need to do is left outer join two text files to the main database table. After sorting the dataset if there are more then 10 records in a group I need to count how many records match each of text files. If the largest replacement is enough to reduce the group count to <=10 then move on otherwise repeat the step for the second largest replacement count.
Re-sort the dataset then if any groups still have a count greater then 10 change everything past 10 to "Other".
One Star

Re: One output as multiuple inputs?

Hi,
To accomplish this I think I need the "intra-group" ranking function for the "Other" peice.
As for the first steps what should I do is maybe:
If (Max($agg1 > 10) {
If ($agg2 > $agg2) {
$firstReplace = $agg1 - $agg2 }
Else {
$firstReplace = $agg1 - $agg2 }
If ($firstReplace < 10) {
#do the first replace only }
Else {
If ($agg2 < $agg2) {
$secondReplace = $firstReplace - $agg2 }
Else {
$secondReplace = $firstReplace - $agg2 }
#do the first and second replacments
If ($secondReplace > 10) {
If ($ranking > 10) {
#replace all ranks > 10 with 'Other' }
}

Else {
#do no replacements }

Attached is an image of the type of workflow operation I would like to do with this multiple input/output questions...

Does that make any sense? Any help would be greatly appreciated. Please ask about anything that I didn't explain properly in the above.
Many Thanks,
Jay
Employee

Re: One output as multiuple inputs?

Jay, what you want to do seems a bit "complicated" to me. In addition to the description of your job, we also need some data example (input and output).
One Star

Re: One output as multiuple inputs?

Hi,
The relevant data attributes include three strings (name, type, subtype) as well as numeric field (value). There are actually 15 fields in all. For each NAME there are more than 1 TYPE. The tasks described above only apply to the TYPE = "Country".
The input is something like:
NAME TYPE SUBTYPE VALUE
ProductA Country France 0.9
ProductA Group GroupA 0.75
ProductA Country Canada 0.05
ProductA Country United States 0.05
ProductA Group GroupC 0.25
The output would be something like:
NAME TYPE SUBTYPE VALUE
ProductA Country France 0.9
ProductA Group GroupA 0.75
ProductA Country North America 0.1
ProductA Group GroupC 0.25

This data example is fairly small though there could be any number of countries.
The desired output dataset contains 10 records per NAME & TYPE=?Country? and possibly an 11th record with the subtype "Other" if necessary.
The consolidation rules have a few extra pieces above and beyond what has already been described as well but I'll stick with the part that was already posted.
The requirement was to have as little as possible in "Other" so two possible consolidations have been chosen. Overall it is desirable to have as few consolidations as is possible.
So for all products that have >10 countries we need to count how many countries fall into each of the two aggregations. Would one aggregation be enough to get it to 10 or less? If it does then do the one only else do the second as well. If the second is applied we could still be over 10 so then the smallest (ranked by VALUE) N records past 10 should be aggregated to "Other".
One thing of note is that if either one of the two consolodations would get the total number <=10 then use the one with the least number of Countries in it (i.e. that maximizes the number of rows (<=10)) should be chosen.
During this whole process TYPES other than "Country" are ignored but need to remain in the final output.
Is that enough information? Please let me know what else you need.
Many Thanks,
Jay