File polling till specific time

Highlighted
Six Stars

File polling till specific time

Hi All,

I have a requirement for polling a directory for files and I am using tWaitforFile Components but facing one issue. Actually tWaitforFile provides option for seconds between iterations and no of iterations to exit out of infinite loop but I want to do file polling till specific time, suppose if job starts at 8 then till 10 I need to do file polling. 

 

If would be great if anyone worked on the same scenario and share their approach.

 

Highlighted
Nine Stars

Re: File polling till specific time

Hi,

 

In this scenario, you could use the tLoop component, tFileExists & tSleep.

 

However generally, I wouldn't recommend a Talend start and stay running, looping for an indefinite period of time - could you not just trigger the job to run every few minutes externally (from CRON or something) between 8 & 10?

 

Regards David
Dont forget to give Kudos when an answer is helpful or mark the answer as the solution.
Highlighted
Forteen Stars

Re: File polling till specific time

as variant - combination

 

tLopp + tSleep + compare time from start:

 

tLoop.PNG

 

at the begin in tJava save StartTime:

context.StartTime=TalendDate.getCurrentDate();

and compare it with current time for each iteration:

TalendDate.compareDate(TalendDate.getCurrentDate(),TalendDate.addDate(context.StartTime,2,"HH")) <=0

similar logic could be about absolute time

 

and also you can add in tLoop criteria && additional parameter - is FileReceived and put this to 1 for exit loop earlier

-----------

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog