One Star

File based job trigger for deploying talend open studio jobs.

Is it possible to achieve file based triggering when we deploy standalone jobs on linux environment?We are working on integrating some ETL tasks(which includes some transformation and file movement to ftp), with the existing framework.The output file generated by the first process will be the starting point of Talend jobs.We are using Open studio,so I was planning to take out the job and schedule it in linux crontab. It will be time based scheduling,what I am looking for is a file based trigger.One option which I can think of is to schedule the job to run every one or two minutes after including a twaitforfile component.This way the job will be triggerred multiple times but will actually run once the file arrives.Another option,which is quite similar to the above,is message queues by making use of tmominput and listening to the message from an incoming queue.I have to try these two methods to see how it works out.Is there any better approach which I can make use of?
8 REPLIES
Moderator

Re: File based job trigger for deploying talend open studio jobs.

Hi,
If we understand your requirment very well, are you looking for a scheduling that file based triggering?(file creation,file delete... )?
Here is a file trigger feature in Talend Administration Center.
Please have a look at document about:TalendHelpCenter:How to add a File trigger.
So far, talend provides community users with a free trial Talend Enterprise Data Integration product.
You can download it freely from:https://www.talend.com/enterprise-data-integration
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: File based job trigger for deploying talend open studio jobs.

Hi Sabrina,
Thanks for the reply.However I am using Talend open studio,not the enterprise version.
I was looking for some methods to trigger a job based on the arrival of file in linux environment where I have deployed the code.
Regards,
Sunil
One Star

Re: File based job trigger for deploying talend open studio jobs.

Hi,
can we trigger the job based on email arrival ?
For us source is email attachment, when new email is arrived then Talend job should start to process further.
Thanks, 
Moderator

Re: File based job trigger for deploying talend open studio jobs.

Hi Hanuman G,
Here is a component TalendHelpCentertWaitForFile which is used to put the component connected with it in waiting state. It then triggers that component when the defined file operation occurs in the specified directory. 
The work flow should be: 
Parent job: tPop(fetches your email messages from email box)-->trunjob
Child job: twaitforfile(trigger your job when an email file occur in the specified directory)-->your further processing job
Let us know if it is Ok with you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: File based job trigger for deploying talend open studio jobs.

Hi Sabrina, 
thanks for your suggestion, but we would like to schedule the Job in TAC and it should be like, when a new email arrived then immediate this Talend Job(Parent Job) should run.
Query is how the scheduled Job should know whether email arrived or not?
Any suggestion how to schedule this Parent Job in TAC based on email arrival?
Thanks,
Moderator

Re: File based job trigger for deploying talend open studio jobs.

Hi,
You are using talend subscription solution?
If you want to execute a task on a file-based event, such as the presence, creation, modification of the file, you can use the File trigger. 
Please take a look at TalendHelpCenter:How to add a File trigger for details.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: File based job trigger for deploying talend open studio jobs.

Hi,
You are using talend subscription solution?
If you want to execute a task on a file-based event, such as the presence, creation, modification of the file, you can use the File trigger. 
Please take a look at TalendHelpCenter:How to add a File trigger for details.
Best regards
Sabrina

My Customer is using subscription solution, we need to know email trigger process is available in TAC as explained above?
thanks,
One Star

Re: File based job trigger for deploying talend open studio jobs.

I dount if you can use email based triggering.