how to extract the data from azure blob based on last run date in a file?

Six Stars

how to extract the data from azure blob based on last run date in a file?

Hi All,

 

 

first extracting the full data from azure and also capturing the last run date in a file.

eg : extracted data which are in the folder 2018-10-07 to 2018-10-07.

 

now i need to increment the data extraction from the 2018-10-07 to current date -1 for the next run.

 

Please let me know how can implement the job.


Accepted Solutions
Community Manager

Re: how to extract the data from azure blob based on last run date in a file?

In the end of job, store the job execution date in somewhere such as a file or database, in the beginning of job, read the last execution date from the file or database.
Let me know if you need more guides.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

View solution in original post


All Replies
Community Manager

Re: how to extract the data from azure blob based on last run date in a file?

In the end of job, store the job execution date in somewhere such as a file or database, in the beginning of job, read the last execution date from the file or database.
Let me know if you need more guides.

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

View solution in original post

Six Stars

Re: how to extract the data from azure blob based on last run date in a file?

i have stored the run date in the file.. but when i store it in csv file date format is changing from 2018-10-09 to 09-10-2018.

i Need to store it in 2018-10-09 format itself.

 

 

Please let me know how to fetch the date from the file and pass the date to other component to extract the data from the run date to till current date -1.

 

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Talend Cloud Developer Series – Fetching Studio License

This video will help someone new to using Talend Studio get started by connecting to Talend Cloud and fetching the Studio License

Watch Now

Talend Cloud Developer Series - Introduction

The Talend Cloud Developer Series was created to give you a solid foundational understanding of Talend’s Cloud Integration Platform

Watch Now

Talend Cloud Available on Microsoft Azure

An integration platform-as-a-serviceto help enterprises collect, govern, transform, and share data from any data sources

Watch Now