Dynamic query using timestamp

Four Stars

Dynamic query using timestamp

New to this forum and tool. 

I have been able to generate a simple job to read from mySQL and extract it into CSV.

However the source side data being huge I need to extract the same data in different date ranges like "date between <one_date> and <another_date>".

I would however like to set these date ranges in a table / file which can then be picked up in a loop  to execute for a different date range every time till the end of record / file is reached. The CSV will be created in the append mode.

Any pointers to a similar question asked previously would be helpful.

 

-A

Ten Stars

Re: Dynamic query using timestamp

If you have your start date and end date in a data source (file or table both work), you can use a tFlowToIterate to execute a subjob once for each row in your input.  The values for each row are converted to global variables which you can reference in the subjob.  Read up on tFlowToIterate here:
https://help.talend.com/#/reader/wDRBNUuxk629sNcI0dNYaA/WqJCkTE5~5DDaSFvtNXZnQ

Four Stars

Re: Dynamic query using timestamp

Created a flow using the documentation link.

However getting the error as seen below. The date format seems to be incorrect.

The date format in the file is 'yyyy-MM-dd' that is used to set the global variable.

 

Error.png

Ten Stars

Re: Dynamic query using timestamp

You will likely want to format the date to a value accepted by your database. TalendDate.formatDate("yyyy-MM-dd", <variable goes here>)

Ex:

TalendDate.formatDate("yyyy-MM-dd",row6.DATE)

Drop your global variable in as the second argument and it should work better.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog