One Star

How to auto run job when file is available in folder

Hi,
I want to know how to auto run the job to read file (file available in folder) and load into database. 
Thanks.
9 REPLIES
Community Manager

Re: How to auto run job when file is available in folder

Hi
You can use twaitforfile to monitor the folder and trigger the business processing once a file is created/updated/deleted. 
Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: How to auto run job when file is available in folder

Thanks for your feedback...
What is difference between tFtpFileExist and twaitforfile?
My situation :
I have a job that read file from ftp in folder and load the data into database.
The job will run using window task scheduler. Or i can use other alternative to auto run the job when file is available in the ftp folder without set timing in task scheduler trigger?
Community Manager

Re: How to auto run job when file is available in folder

tFtpFileExist is used to check if a file exists on FTP server, twaitforFile is used to monitor a local folder where the job is running. I want to know if the job will be executed on the same machine where FTP server is installed?
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: How to auto run job when file is available in folder

The job will be executed in different machine. 
Community Manager

Re: How to auto run job when file is available in folder

So, you need to check if the file exists on ftp server using tFTPFileExist, download the file from ftp to local system, read the file and load the data to database. Schedule the job to run once in a while using window task scheduler or other scheduler tools. 
If you are using enterprise subscription product, you can schedule the tasks in Job Conductor. 
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: How to auto run job when file is available in folder

Currently my job use tftpfileexist to check the file is exist or not.
For example,
File exist at FTP folder on 15/02/2017, 9 am.
No file exist at FTP folder on 16/02/2017, 11 am.
If i schedule use task scheduler set trigger as daily, 10 pm. 
15/02/2017 - true (Job will running and data will loaded at 15/02/2017, 10 pm)
16/02/2017 - false (No data loaded because no file exist)
My question is, can we run the job without waiting 15/02/2017, 10 pm. Once file is available at 9 am.
Or i need to schedule every one hour?
Twelve Stars

Re: How to auto run job when file is available in folder

Currently my job use tftpfileexist to check the file is exist or not.
For example,
File exist at FTP folder on 15/02/2017, 9 am.
No file exist at FTP folder on 16/02/2017, 11 am.
If i schedule use task scheduler set trigger as daily, 10 pm. 
15/02/2017 - true (Job will running and data will loaded at 15/02/2017, 10 pm)
16/02/2017 - false (No data loaded because no file exist)
My question is, can we run the job without waiting 15/02/2017, 10 pm. Once file is available at 9 am.
Or i need to schedule every one hour?

Because it different servers short answer is - You must run Job regular
More longer - You can design bypass solutions, such as:
- if it same network (corporate) - You can mount NFS folder and look for it as local
- if it external server, but You can run Jobs on it, You can run daemon on remote server which will call Talend Job by ssh or send message when new files coming and on local server monitor messages and fire Job when new message arrive (it much longer story, but both work)
-----------
Community Manager

Re: How to auto run job when file is available in folder

My question is, can we run the job without waiting 15/02/2017, 10 pm. Once file is available at 9 am.
Or i need to schedule every one hour?

Yes, you need to schedule the job to run more often, so that the file will be read and process asap.
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: How to auto run job when file is available in folder

Thank you for your help....