One Star

Need advice for passing data to a subjob using contexts

Hello,
I am a new talend user. I have done some searches and I could not find exactly my question.
I am designing a parent job, and 2 subjobs. First, let me describe the parent job:
The parent job will read an input file, parse this data and then query an oracle database using this input. Then, we filter the rows, and proceed to subjob A or subjob B depending upon the filtering.
Subjob A and Subjob B perform different final actions, but the overall idea is the same:
Query additional data depending upon which subjob, and then perform a final action.
I need both subjob A and subjob B to be reusable such that they could potentially be called from different jobs.

Thus, I have arrived at these choices:
A.) Passing variables via context. I understand this is the suggested way to do it with the free version TOS.
B.) Having the parent jobs generate an output file and then have the subjobs read this output file.
C.) I cannot use option C, but I saw mention to Jobjet (joblets maybe?) which allow the output of a parent to be passed to the child job directly. As I understand it, this is for a licensed version, which I do not have. Thus I cannot consider this.
As I understand it, the advantages to using point 1 are that it would be faster, and this is the usual way I see this suggested on the forums.
The advantage to using option 2 is that it allows for subjob A and subjob B to be reused without having to pass the contexts each time.
Here are some questions:
1. Is there any way to manage this? If I use the input as contexts, I cannot reuse subjob A or subjob B without passing them the variables as contexts each time (is this assumption correct???). Additionally contexts will be very painful, as I will be passing upwards of 200+ variables for some subjobs.
2. Is there a way to quickly map contexts using java rows (the only way I know of is context.X = input_row.X, etc)?
3. How bad is the performance hit for creating an output, then having the subjob read that as an input? Using this would allow me to manage the reuse.

I appreciate any input and suggestions.
Thanks.

  • Data Integration
1 REPLY
Moderator

Re: Need advice for passing data to a subjob using contexts

Hi,
Here is a KB article about Passing+a+value+from+a+parent+Job+to+a+child+Job, maybe it will be useful for you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.