I have CSV files stored in Azure blob storage and want to process it into a SQL DB. Right ow I have this working like this:
tAzureStorageGet -> [Temp Local File System] -> tFileList [iterate]-> tFileInputDelimited -> etc.
Is there a way to skip the local file system and tFileList steps and process the file directly out of Azure storage?
So far, talend doesn't support for transferring data by air.
You have to save your Azure storage files in local firstly and then load them into a SQL DB.
Thanks Sabrina. I have limited TEMP storage so would need to iterate through the items in Azure, download each locally, process, remove and repeat. Could you give me a hint as to how to achieve that please? I'm not sure how to use 'AzureStorageGetList" as an input into "AureStorageGet"
Here is a scenario about:TalendHelpCenter:Retrieving files from a Azure Storage container. Hope it will help.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.