Shared variable between 2 jobs

One Star

Shared variable between 2 jobs

Why does this simple thing seem impossible?
a) I have a PARENT job that calls SUBJOB1, then SUBJOB2 after SUBJOB1 is done

b) All 3 jobs share the same context with a context variable called "listOfThings" initialized to blank

c) SUBJOB1 appends data to the "listOfThings" context variable. 

d) When SUBJOB2 reads "listOfThings", it is blank

e) I tried this with "globalMap" as well to no avail, it appears "globalMap" is only global to the current job and re-created for every job and hence has no data in it on SUBJOB2

f) If  context var "listOfThings" is initialized to "1" this value is available to all subjobs, HOWEVER if any subjob mutates this context variable, that mutation does not carry over to subsequent subjobs that are executed.

This just seems like a very simple thing to me, to allow multiple jobs to interact with one another via some sort of thread-local variable or true "global" variables that span a job and any subjobs it invokes.
Confused and in need of a solution to this.
Five Stars

Re: Shared variable between 2 jobs

Here's one approach that you can take.
See the section "Returning Values through Context"
One Star

Re: Shared variable between 2 jobs

Simplest solution seems to be just create a "routine" backed with a ConcurrentHashMap that exposes get/put methods. Since an entire job is in the same Classloader this static instance is available to all jobs in the process.
Community Manager

Re: Shared variable between 2 jobs

Context variables and globalMap variables are only available to components inside individual jobs automatically. You can send then to child jobs (as I guess you are aware) pretty easily, but sending the values back to the parent is not so nice. The way they work makes sense (ie changing them in child jobs not automatically changing them in parent jobs), but it is annoying having to manually pass the values back if you want the values to change with the child jobs. 
Your solution (the ConcurrentHashMap) is what I use if I need to do this. I think it could be handled in a better way by having an option on the tRunJob component to allow the contexts that are sent to the job to be passed back to the parent. Maybe this could be an enhancement Talend?
One Star

Re: Shared variable between 2 jobs

Yes the whole thing is annoying and completely non-intuitive the way they have designed it.
Create yourself a Routine with a static map member and just read/write from that. Shared classloader will ensure that all jobs/subjobs in one job access the same data. No clicking on buttons and moving variables around etc. i.e. you just do: MyRoutine.set(varname,value) and MyRoutine.get(varname)
Four Stars

Re: Shared variable between 2 jobs



Can you provide example of this concurrentHashMap routine with get / set methods?



Two Stars

Re: Shared variable between 2 jobs

Create a new Routine (under code in the repository list), call it UniversalMap, and paste this in there:

package routines;

import java.util.concurrent.ConcurrentHashMap;

public class UniversalMap {

    private static ConcurrentHashMap<String, Object> map = new ConcurrentHashMap<>();

    public static <T> T put(String key, T value) {
        return (T) map.put(key, value);
    public static <T> T get(String key) {
    	return (T) map.get(key);


Then you can use it like this:



String who = UniversalMap.get("hello");



Seven Stars

Re: Shared variable between 2 jobs

This is my version of the sharedMap infrastructure.

1. Add a context variable sharedMap to each job

2. Create a joblet:


3. Add this joblet to each job, and link it using an onComponentOK link at the start of the job

4. In the tRunJob components, pass the sharedMap context variable the value context.sharedMap

5. In PARENT, put the following code in a tJava:

(( context.sharedMap).put( "listOfThings", "" );

6. In SUBJOB1, put the following code:

(( context.sharedMap).put( "listOfThings", (String)((( context.sharedMap).get("listOfThings")) + my.thing );

...where my.thing is whatever you want to add

7. In SUBJOB2, use the following wherever you want to use the listOfThings:

(String)((( context.sharedMap).get( "listOfThings" ))


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables


How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration


6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend