Dynamic query using timestamp

Four Stars

Dynamic query using timestamp

New to this forum and tool. 

I have been able to generate a simple job to read from mySQL and extract it into CSV.

However the source side data being huge I need to extract the same data in different date ranges like "date between <one_date> and <another_date>".

I would however like to set these date ranges in a table / file which can then be picked up in a loop  to execute for a different date range every time till the end of record / file is reached. The CSV will be created in the append mode.

Any pointers to a similar question asked previously would be helpful.

 

-A

Ten Stars

Re: Dynamic query using timestamp

If you have your start date and end date in a data source (file or table both work), you can use a tFlowToIterate to execute a subjob once for each row in your input.  The values for each row are converted to global variables which you can reference in the subjob.  Read up on tFlowToIterate here:
https://help.talend.com/#/reader/wDRBNUuxk629sNcI0dNYaA/WqJCkTE5~5DDaSFvtNXZnQ

Four Stars

Re: Dynamic query using timestamp

Created a flow using the documentation link.

However getting the error as seen below. The date format seems to be incorrect.

The date format in the file is 'yyyy-MM-dd' that is used to set the global variable.

 

Error.png

Ten Stars

Re: Dynamic query using timestamp

You will likely want to format the date to a value accepted by your database. TalendDate.formatDate("yyyy-MM-dd", <variable goes here>)

Ex:

TalendDate.formatDate("yyyy-MM-dd",row6.DATE)

Drop your global variable in as the second argument and it should work better.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog