Iterate a job multiple times with different input for each run

Six Stars

Iterate a job multiple times with different input for each run

Hi all,

 

I have a job that makes a download of folders and sub folders based on a variable that contains the current date.

 

Now, based on the current date, I want to download the same structure for each day based on a range of time.

 

E.g.: today is 2018/09/24 and I'm interested on the folders from today up to a week ago.

1) I start from today so I download all the folders (containing files) for 2018/09/24

2) yesterday was 2018/09/23 so I'll download all the related folders (and subfolders)

3) then the same as above but for the day before and so on until a week

 

I have the single job, how can I iterate it with a range of time?

 

I started evaluating the tForEach, but I'm not sure that is the right direction.

 

Any suggestion is appreciated.

 

Thanks,

Nick

Highlighted
Eight Stars

Re: Iterate a job multiple times with different input for each run

I would use a tJavaFlex.
Six Stars

Re: Iterate a job multiple times with different input for each run

hi navds, thank you for your hint. But my question would be: why and how to apply it? 

 

This is the job flow I have in place with a short description:

unarchive_job.PNG

 

 

I want to execute it n times, where n is equal to the number of days contained within a range of time. According to the example above, it would be 7 days. But in general it could be a month, or 2 weeks...

Eight Stars

Re: Iterate a job multiple times with different input for each run

  • Why ? Because it is simple
  • How ? Like this:

Screenshot from 2018-09-25 11-59-10.png

You just need to replace the tLogRow with tRunJob where you pass the date as context.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now

How to Modernize Your Cloud Platform for Big Data Analytics With Talend and Micr...

Learn how<SPAN>to modernize your Cloud Platform for Big Data Analytics with Talend and Microsoft Azure</SPAN>

Read