I need to implement a job that will watch file for specific interval and generate notification.
Schedule the task with the desired interval (using TAC or an external scheduler) or use tSleep to introduce the wait strategy into the job.
In this case you should anticipate a condition to break the infinite loop.
all what suggested by @TRF, also:
- look for ESB, where You can create route which monitor folders
- Use remote agents with could ship files (or file content)
last also allow tail file in "real-time" to message queue
This is what I need to implement
System should perform below validations for file reception
- File should be received by stipulated (configured time) time for particular Issuer
- For each issuer two files should be present – Main and Meta
If file not received by stipulated time, then system should retry up to configured number of times
- System should retry after configured duration
Can you please elaborate how to implement this. The directory that needs to be scanned will have multiple files coming , I need to scan this folder, read these files and do validations and processing.
Appreciate your co-operation and timely help.
- Schedule Job as usual - by any scheduler
- tLoop with number of iteration what You want to re-try
- iterate to next steps:
- tFileExists for both of files
- trigger IF exists both - run all steps and exit Loop
- trigger IF not exists both - tSleep for predicted number of seconds
- after tSleep if number of current iteration == maximum defined - notify about unsuccessful execution
Above - simple sequence for single issuer, but You can extend it for iterate thru all defined issuers
Join us at the Community Lounge.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend