One Star

Delegating tRESTRequest output flows to separate jobs using tRunJob

Hi all
I have a use case where I want to use Talend Open Studio for ESB to create a proxy REST API for an existing backend REST API. For each call to the proxy API, it will perform one or more API calls to the existing and perform the necessary transformations.
As the required logic for each call to the proxy API can be quite complex, I want to encapsulate the logic required for each proxy API call in a separate job. At first sight, it seems that the tRunJob component is the component I need to achieve this.
I have done a lot of reading, including this thread: . However, some things are still unclear to me.
In my current setup (without tRunJob), I have a simple flow with a tRESTRequest representing the proxy API, which routes an output flow to a tXMLMap. The result of the tXMLMap goes to a tRESTClient that calls our existing backend and sends its response to a tRESTResponse component.
Now, I have created a job with a tRESTRequest for which each output flow calls a different tRunJob and I want to implement the child jobs. I am now running into some issues:

Which component should the jobs called by tRunJob start with? It seems I can't use the tXMLMap as the first component, so I'm using a tLogRow for now.
It seems that passing data from parent job to child job typically happens using context variables. However, it seems that this does not support passing Documents. Is there any way to pass a Document from a parent job to a child job?

Any help is very much appreciated
Kind regards
Maarten
4 REPLIES
One Star

Re: Delegating tRESTRequest output flows to separate jobs using tRunJob

It seems that I cannot post links. When I say "I have done a lot of reading, including this thread: ", I mean the thread with ID 44352.
One Star

Re: Delegating tRESTRequest output flows to separate jobs using tRunJob

Hello Maarten
maybe we cannot help you with all the problem at once, but lets try
- As a first component in the subjobs we use the tFixedFlow to pass the context variables into the flow (as a value you may enter context.variableName)
- is there a reason why the context variable (passed into the tRunJob invocation) couldn't be a Document? (I cannot recall if it is supported or not). Still, maybe not efficient, you may serialize / deserialize the XML (so you pass it as a string).
Carpe diem
         Gabriel
One Star

Re: Delegating tRESTRequest output flows to separate jobs using tRunJob

Hi Gabriel
Thank you for your answer.
Regarding passing a Document as a context variable: Document is not a supported context variable type. 
I am now trying to pass a Document as an Object and convert it to a Document again in the child job. I have no idea if this is the "right approach", but we'll see how far I get.
Kind regards
Maarten
One Star

Re: Delegating tRESTRequest output flows to separate jobs using tRunJob

Update: I got it working!
This is how I am now passing the body (of type Document) of a tRESTRequest output flow from a parent job to a child job (and returning the resulting body (of type Document) from the child job to the parent job: 

Define a context variable with type Object in the child job
In the parent job, connect the output flow from your tRESTRequest to a tRunJobComponent. In the tRunJob component, select the Child job, add the context variable of the child job under "Context Param" and pass the body of the tRESTRequest output flow to that variable. On the "Advanced settings" tab, make sure that "Propagate the child result to the output schema" is checked.
Connect the output of the tRunJob component to your tRESTResponse.
In the child job, create a tFixedFlowInput and edit its schema to contain a column "body" of type Object. Make it take the value of the context variable to fill this column. Connect the output of the component to a tJavaRow component. Edit the schema of the tJavaRow component so the input has one column "body" with type Object and the output has one column "body" with type Document. The code for the tJavaRow is the following:
output_row.body = (Document) input_row.body;

I then call our existing backend, giving me a message with a single column "body" of type Document. I pass this to a tBufferOutput at the end of the child job, making it available at the tRunJob output in the parent job.

I hope this may help someone someday Smiley Happy