Six Stars

Find talend job process name at operating system level

Hi

I have a job which has to run throughout the day, if it aborts for any reason, it has to be resubmitted automatically, In other ETL tools like Datastage, I would be able to find the ETL job process name at OS level using ps-ef, when i tried that for talend in windows environment using tasklist command, i'm not able to find that Talend job name process which runs at the backed. I know I can create another job which checks if a process is running  but I'm trying to create a non Talend based solution and i don't want to export the job and run it outside TAC.. so how can find at OS level if a job is running. 

 

Thanks

MJ

 

Tags (1)
2 ACCEPTED SOLUTIONS

Accepted Solutions
Fifteen Stars

Re: Find talend job process name at operating system level

The answer to your question is the following command on the machine that is running your job....

 

jps -v

On another note, I think you should reconsider your process here. A Talend job is not really suitable to be running all day non-stop. They are meant for batch. You can run a Talend job 1000s of times a day, no problem. If you try to run one continuously for the whole day you will likely end up with memory issues after a while. It is not what Talend DI jobs have been designed for. Your problem sounds to me to be more aligned with a real time (or near to real time) solution. It sounds like a Talend Route (ESB) would be a better fit.

Rilhia Solutions
Fifteen Stars

Re: Find talend job process name at operating system level

The first way that springs to mind would not be much of a change to your job structure, but would make it a little more resistant to issues. You could use a Windows scheduler (or Cron on Linux) to run your orchestration job every hour. I assume that the child jobs need to run strictly sequentially (with maybe some interaction with data processed?). As I said, this would not be a massive change for you. The big issue here is solely having a Talend job running as a scheduling device. I assume you have had issues with doing this, hence the original question about monitoring whether it is still running.

Rilhia Solutions
3 REPLIES
Fifteen Stars

Re: Find talend job process name at operating system level

The answer to your question is the following command on the machine that is running your job....

 

jps -v

On another note, I think you should reconsider your process here. A Talend job is not really suitable to be running all day non-stop. They are meant for batch. You can run a Talend job 1000s of times a day, no problem. If you try to run one continuously for the whole day you will likely end up with memory issues after a while. It is not what Talend DI jobs have been designed for. Your problem sounds to me to be more aligned with a real time (or near to real time) solution. It sounds like a Talend Route (ESB) would be a better fit.

Rilhia Solutions
Six Stars

Re: Find talend job process name at operating system level

Hi
Thanks for responding, to be more specific,I have a main job (Orchestrate job) that has to run from 5.00 AM to 4.30 PM, That orchestrate job triggers a set of 5 jobs sequentially for every 1 hour. The thought behind the question was, I don't want any manual intervention if the main orchestrate job abort, so how do you think I can better design this without the memory issue? please

thanks
MJ

Fifteen Stars

Re: Find talend job process name at operating system level

The first way that springs to mind would not be much of a change to your job structure, but would make it a little more resistant to issues. You could use a Windows scheduler (or Cron on Linux) to run your orchestration job every hour. I assume that the child jobs need to run strictly sequentially (with maybe some interaction with data processed?). As I said, this would not be a massive change for you. The big issue here is solely having a Talend job running as a scheduling device. I assume you have had issues with doing this, hence the original question about monitoring whether it is still running.

Rilhia Solutions