Can I create a Job with multiple paths from a single source to the same target?


Creating a subjob structure with multiple paths from a single source to a single target, as shown in the capture below, is not allowed in the Talend Studio.


As a workaround, you can solve this problem by designing the job in one of the following ways:


Solution 1

Solution 1 consists of doubling the input flow to avoid the flow circle.

  1. Duplicate the input flow part that is composed of tFileInputDelimited_1 and tReplicate_1.

    In this job, tFileInputDelimited_2 and tReplicate_2 are a simple copy and paste of tFileInputDelimited_1 and tReplicate_1, thus performing the same processing operation.

  2. Configure the properties of each component for both input flows. As they're duplicates, both input flows read the same source file and therefore are configured the exact same way in their respective Properties view.


This Job design performs the exact same operation and let you get the same result as the unauthorized job described above.

However, if your input flow is complex, contains numerous components, and/or processes large sets of data, you may notice a significant decrease of performance. The reason for this performance issue is that the Job spends twice as much time performing the same processing. Therefore, for complex input flows, you should consider Solution 2, described below.


Solution 2

The second solution works out the output flow part of the Job. The point is to store the result of the first Subjob (input) in a temporary location: either in a file or in memory (cache).

  1. Use a tFileOutputDelimited or a tHashOutput component to store the result in place from tMap_1 or tMap_2.
  2. Then read the data in the next Subjob, from the temporary file using a tFileInputDelimited component, or from memory using a tHashInput component.

    The job example below caches the result into memory:


    In this case:

    • tHashOutput_1 is used to cache the result out from tMap_1 into memory.
    • tHashOutput_2 is used to cache the result out from tMap_2 into memory.
    • In order for the data to be retrieved from memory, the tHashInput_1 component must be linked with the tHashOutput_1 component, and tHashInput_2 with tHashOuput_2, respectively. This should be configured in the respective component Properties view.


    Solution 2 performs the same operation and provides the same result as the initial unauthorized job.

Version history
Revision #:
7 of 7
Last update:
‎04-04-2019 03:25 AM
Updated by:
Labels (2)
Five Stars

Though these workarounds do the trick in most cases, other ETL software does not suffer from such weird shortcomings. Thus, I would appreciate you fixing it.

Community Manager

Hi fuggi  

Thanks for your comment. This is a known limitation in Talend studio.