Iterate a job

Five Stars

Iterate a job


I need to extract data from big query for last 2 years. 

I populated a table with dates with last 2 years. 

Now I want to read date from this table and get daily data from big query for last 2 years.

What would be the best way iterate this job?

PS. Table_date_range in big query failed because of too much data. 


Community Manager

Re: Iterate a job

If I understand your request well, you need to iterate each date and pass it to the query as a parameter to query only the data based on the current date.
Take a look at the tFlowToIterate component documentation and learn this component. eg:
Read date from data source--main(row1)--tFlowToIterate--iterate-->tBigQueryInput....

on tBigQueryInput, you can access the current value of date_column with this expression:


Talend | Data Agility for Modern Business