once-daily execution of a Talend job, but at a variable time

Six Stars

once-daily execution of a Talend job, but at a variable time

I have a Talend job (call it A) that I want to run when an Oracle stored procedure (not executed by Talend) has completed. I have an Oracle control table that I can check to see if the Oracle SP has finished.
I am imagining having my Talend job A run on a schedule - perhaps every 15 minutes - and the first thing that it would do is determine if A has already executed today. If it has already run today it would stop execution immediately. If it has not yet run today, it will check to see if the Oracle SP has completed, and if not, it will stop execution immediately. This should result in Talend job A only executing once a day, and only after the Oracle SP has completed.
Is there a more elegant way to accomplish this?
Thanks.
Community Manager

Re: once-daily execution of a Talend job, but at a variable time

Hi
I would like to create another job let's call it myMain job, and schedule this job to run every some time, perhaps every one day, the myMain job will call the job A based on the condition that the job A has not executed today and the oracle SP has completed.
For example:
myMain job:
tFileInputDelimited--main--tJavaRow_1--runIf_1-->tOracleInput--main--tJavaRow_2---runIf_2--tRunJob
tFileInputDelimited: suppose you have output a flag that indicates the job A has executed or not today to a text file, read this flag to check if the job A has executed or not.
on tJavaRow_1, put the result to a global variable: (Suppose the value of flag column is true or false)
if(input_row.flag.equals("true")){
globalMap.put("AhasExecuted",true);
}else{
globalMap.put("AhasExecuted",false);
}
Set the condition of runIf_1 as:
!(Boolean)globalMap.get("AhasExecuted")
tOracleInput: read the result of SP, and check it on tJavaRow component,
on tJavaRow_2: (Suppose the value of result column is finished or unfinished)
if(input_row.result.equals("finished")){
globalMap.put("SPHasFinised", ture);
}else{
globalMap.put("SPHasFinised", false);
}
Set the condition of runIf_2 as:
(Boolean)globalMap.get("SPHasExecuted")
tRunJob: call job A.

Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: once-daily execution of a Talend job, but at a variable time

I have an Oracle control table that I can check to see if the Oracle SP has finished.
>> As you said, you have oracle control table which sets say flag if the SP has finished...
then
toracleinput--> RunIf (condition on flag) -->your job
Does this helps you?
Thanks
Vaibhav
Six Stars

Re: once-daily execution of a Talend job, but at a variable time

Thanks all. This is helpful.
But I think I didn't do a good job of explaining that I want Job A to run as soon as possible (doesn't have to be immediate - 5-15 min is fine) after the completion of the Oracle SP. So the part I was unsure about was whether I should:
(a) run Job A on a 15 min schedule (where it obviously only Does Stuff based on the correct conditions (Oracle SP has completed and Job A has not yet run today)
(b) change Job A to run once a day for a say 23hr 15 min duration, which has a sleep, and checks for the conditions and then Does Stuff when the conditions are right
Basically what is the most elegant way to run a Talend job at an uncertain time, triggered by some other event (a record in an Oracle table, in my case).
I figured someone is already doing something like this in production.
Thanks for your detailed response shong. Using your terminology, I am wondering how best to execute/schedule myMain. I guess run myMain on a 15 min interval? (that is my scenario (a) above)
Community Manager

Re: once-daily execution of a Talend job, but at a variable time

Hi
I want Job A to run as soon as possible (doesn't have to be immediate - 5-15 min is fine) after the completion of the Oracle SP

If you want Job A to run as soon as possible, you can use a tInfiniteLoop component to check if Job A has already executed or not every 5-15 mins, if Job A has executed one time today, use a tDie to exit the loop.
myMain job:
tInfiniteloop
|
iterate
|
tFileInputDelimited--main--tJavaRow_1--runIf_1-->tOracleInput--main--tJavaRow_2---runIf_2--tRunJob
--runIf_3-->tDie
Set the condition of runIf_3 as:
(Boolean)globalMap.get("AhasExecuted")
Schedule the myMain job to run at a specified time of each day (for example, 8 o'clock per day), once myMain job starts to run, it will loop to check if Job A has already executed or not every 5-15 mins.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business