Four Stars

Schedule talend job using cron tab

Hi,

i want to schedule the talend job (.sh) file using cron tab by creating dependencies between jobs like after successful execution of one job only other job should execute. Can any one tell me how to do it.

3 REPLIES
Nine Stars

Re: Schedule talend jon using cron tab

 

Use this with nohup to start in background and return to your prompt:

nohup sh <yourjob>.sh --context_param acontextvariable1='2018-07-21' --context_param acontextvariable2='2018-07-21' --context_param acontextvariable3='clientstable' > /data/log/<yourlogfilename>.log 2>&1 </dev/null &

 

Four Stars

Re: Schedule talend job using cron tab

Hi Dijke,

 

Thanks for your reply. i want to create dependency between two jobs  after success of one job only other job should execute otherwise should not execute.

 

can you explain the command you posted.

 

Nine Stars

Re: Schedule talend job using cron tab

I've already explained the command...
nohup -> background task
sh -> execute <yourjob>.sh
--context_param -> pass value to a context variable like cronStartDate='2018-07-21'
context vars you need to set in your job in order to pass your crontab parameters, in example your startdate and an enddate.
> /data/log/<yourlogfilename>.log 2>&1 & -> root stdout and error to a log file and return to unix prompt

For understanding crontab (Cron Table), google it please to get a better understanding also to implement dependencies.

However, a sidenote: I prefer to create my dependencies (provenance) in my job flow not with crontab.
And pass log events (tlogcatcher) to my elastic search environment and visualize it with Kibana.
The generated logfile (from above) is also picked up by logstash and inserted into eleastic search...

Dont integrate to deeply with crontab... keep it at just triggering a job. But it highly depends on your daily operations and maintenance strategy/policy by your company.

Look at ELK stack : ElasticSearch Logstash Kibana., not a scheduling tool and has nothing to do with it.