Pass context variable or value from child job to parent job

Four Stars

Pass context variable or value from child job to parent job

Hi All,

I have a screnario where im doing file validation and moving the file from local to hdfs and creating external hive table in one job "job1" (trunjob1) , i am calculating the tot_rec_cnt in that subjob. Upon success or failure of subjob i want to trigger another subjob "job2" (trunjob) and want to pass the tot_rec_cnt from job1 to job2.

I have captured the tot_rec_cnt as a context variable and i want to pass the context to main job so that i can pass the same to my audit job(job2).

My job flow is like:

trunjob1(job1-filevalidation)  ----onsubjobok---- trunjob2(job2 audit job)

 |

onsubjoberror

|

trunjob2(job2 audit job)

I am calculating the tot_rec_cnt in trunjob1(job1) and want to pass the value to audit job. And adding the this, i want my audit job to called either for the success or failure of my file validation job

Kindly help to resolve this.


Accepted Solutions
Community Manager

Re: Pass context variable or value from child job to parent job

There is another way of approaching this. You can set up a routine with static public variables. These will be available across your whole job (parent and child jobs). You can set this up really easily like below....

 

public class StaticVariables {
	
	public static String myString1;
	public static int myInt1;
}

....and access/assign these values using code like below (assuming your routine is called MyStaticVariables).....

routines.MyStaticVariables.myString1 = "Hello";
routines.MyStaticVariables.myInt1 = 100;

System.out.println(routines.MyStaticVariables.myString1);
System.out.println(routines.MyStaticVariables.myInt1+"");

All Replies
Employee

Re: Pass context variable or value from child job to parent job

You should look at the tBufferOuput component which is usually used with tRunJob.

See https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/yh_GCK7WMATxbOSlI_Ck1Q

 

You send the values into the buffer in the child job.  You can link your tRunJob to a tMap or tLogRow or whatever components and read the value sent back by the child job into the tBufferOutput.

Four Stars

Re: Pass context variable or value from child job to parent job

Hi ,
As of now I have done like that only.. I'm passing the value of job1 to buffer component and taking the value using tjavarow component from main job.
But the issue is whenever the job1 is success or gets failed, I want the tot_rec_cnt value to b passed to my audit job.
But as per this design,upon success of trunjob1, the value will be passed,correct?
How I can get the value in case of failure of job too?
Employee

Re: Pass context variable or value from child job to parent job

You should design the child job being called by tRunJob1 in such a way that whether it succeeds or fails, there is always a value that is passed back.  The tPreJob and tPostJob always execute whether your main flow succeed or fails.  

 

You cannot handle the failure and count in a single subjob within the child job.  You need to count, maintain your count, and handle your errors in your subjob.  Then whatever the outcomes the childjob will return the count through the tBufferOutput.

Four Stars

Re: Pass context variable or value from child job to parent job

Hi,
My child job(trunjob1) will read a file from ftp location and then do the file validation and post that I am calling another child job within that which will move the file in hdfs and create a hive external table.
I'm calculating tot_rec_cnt in file validation part.Please let me know how to pass that value upon failure of trunjob1?
Do I need to pass the tot_rec_cnt to some component and attached to tpostjob? Is that what you are telling?
Employee

Re: Pass context variable or value from child job to parent job

Can you upload some screenshot please?

Community Manager

Re: Pass context variable or value from child job to parent job

There is another way of approaching this. You can set up a routine with static public variables. These will be available across your whole job (parent and child jobs). You can set this up really easily like below....

 

public class StaticVariables {
	
	public static String myString1;
	public static int myInt1;
}

....and access/assign these values using code like below (assuming your routine is called MyStaticVariables).....

routines.MyStaticVariables.myString1 = "Hello";
routines.MyStaticVariables.myInt1 = 100;

System.out.println(routines.MyStaticVariables.myString1);
System.out.println(routines.MyStaticVariables.myInt1+"");
Four Stars

Re: Pass context variable or value from child job to parent job

Thanks..Let me try this approach..
Four Stars

Re: Pass context variable or value from child job to parent job

Hi,

Sorry for the delay.. I have attached the main job screenshot where am  calling y file validation job(trunjob1)

and audit job(trunjob2,trunjob3).

I am calculating the tot_rec_cnt in trunjob1 which i need to pass to audit job.Upon success or failure,i want to pass the tot_rec_cnt to audit job.

As of now i have captured the tot_rec_cnt in tbufferoutput and passing the same in tjavarow. But i am afraid that if the file validation job gets failed ,in such situation this approach will fail.

Please advise

Four Stars

Re: Pass context variable or value from child job to parent job

thanks rhall_2_0..it workedSmiley Happy

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

Modern Data Engineering in the Cloud

Learn about modern data engineering in the Cloud

Watch Now

How to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud reg...

Learn how to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud registries

Blog