New to this forum and tool.
I have been able to generate a simple job to read from mySQL and extract it into CSV.
However the source side data being huge I need to extract the same data in different date ranges like "date between <one_date> and <another_date>".
I would however like to set these date ranges in a table / file which can then be picked up in a loop to execute for a different date range every time till the end of record / file is reached. The CSV will be created in the append mode.
Any pointers to a similar question asked previously would be helpful.
If you have your start date and end date in a data source (file or table both work), you can use a tFlowToIterate to execute a subjob once for each row in your input. The values for each row are converted to global variables which you can reference in the subjob. Read up on tFlowToIterate here:
Created a flow using the documentation link.
However getting the error as seen below. The date format seems to be incorrect.
The date format in the file is 'yyyy-MM-dd' that is used to set the global variable.
You will likely want to format the date to a value accepted by your database. TalendDate.formatDate("yyyy-MM-dd", <variable goes here>)
Drop your global variable in as the second argument and it should work better.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend