I have CSV files stored in Azure blob storage and want to process it into a SQL DB. Right ow I have this working like this:
tAzureStorageGet -> [Temp Local File System] -> tFileList [iterate]-> tFileInputDelimited -> etc.
Is there a way to skip the local file system and tFileList steps and process the file directly out of Azure storage?
So far, talend doesn't support for transferring data by air.
You have to save your Azure storage files in local firstly and then load them into a SQL DB.
Thanks Sabrina. I have limited TEMP storage so would need to iterate through the items in Azure, download each locally, process, remove and repeat. Could you give me a hint as to how to achieve that please? I'm not sure how to use 'AzureStorageGetList" as an input into "AureStorageGet"
Here is a scenario about:TalendHelpCenter:Retrieving files from a Azure Storage container. Hope it will help.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema
Pick up some tips and tricks with Context Variables