Shared variable between 2 jobs

One Star

Shared variable between 2 jobs

Why does this simple thing seem impossible?
a) I have a PARENT job that calls SUBJOB1, then SUBJOB2 after SUBJOB1 is done

b) All 3 jobs share the same context with a context variable called "listOfThings" initialized to blank

c) SUBJOB1 appends data to the "listOfThings" context variable. 

d) When SUBJOB2 reads "listOfThings", it is blank

e) I tried this with "globalMap" as well to no avail, it appears "globalMap" is only global to the current job and re-created for every job and hence has no data in it on SUBJOB2

f) If  context var "listOfThings" is initialized to "1" this value is available to all subjobs, HOWEVER if any subjob mutates this context variable, that mutation does not carry over to subsequent subjobs that are executed.

This just seems like a very simple thing to me, to allow multiple jobs to interact with one another via some sort of thread-local variable or true "global" variables that span a job and any subjobs it invokes.
Confused and in need of a solution to this.
Five Stars

Re: Shared variable between 2 jobs

Here's one approach that you can take.
http://www.talendbyexample.com/talend-returning-values-from-subjobs.html
See the section "Returning Values through Context"
One Star

Re: Shared variable between 2 jobs

Simplest solution seems to be just create a "routine" backed with a ConcurrentHashMap that exposes get/put methods. Since an entire job is in the same Classloader this static instance is available to all jobs in the process.
Sixteen Stars

Re: Shared variable between 2 jobs

Context variables and globalMap variables are only available to components inside individual jobs automatically. You can send then to child jobs (as I guess you are aware) pretty easily, but sending the values back to the parent is not so nice. The way they work makes sense (ie changing them in child jobs not automatically changing them in parent jobs), but it is annoying having to manually pass the values back if you want the values to change with the child jobs. 
Your solution (the ConcurrentHashMap) is what I use if I need to do this. I think it could be handled in a better way by having an option on the tRunJob component to allow the contexts that are sent to the job to be passed back to the parent. Maybe this could be an enhancement Talend?
One Star

Re: Shared variable between 2 jobs

Yes the whole thing is annoying and completely non-intuitive the way they have designed it.
Create yourself a Routine with a static map member and just read/write from that. Shared classloader will ensure that all jobs/subjobs in one job access the same data. No clicking on buttons and moving variables around etc. i.e. you just do: MyRoutine.set(varname,value) and MyRoutine.get(varname)
Four Stars

Re: Shared variable between 2 jobs

Hello:

 

Can you provide example of this concurrentHashMap routine with get / set methods?

 

Thanks