tAzureStorageGet directly into tFileInputDelimited

Highlighted
Six Stars

tAzureStorageGet directly into tFileInputDelimited

I have  CSV files stored in Azure blob storage and want to process it into a SQL DB.  Right ow I have this working like this:

 

tAzureStorageGet -> [Temp Local File System] -> tFileList [iterate]-> tFileInputDelimited -> etc.

 

Is there a way to skip the local file system and tFileList steps and process the file directly out of Azure storage?

Moderator

Re: tAzureStorageGet directly into tFileInputDelimited

Hello,

So far, talend doesn't support for transferring data by air.

You have to save your Azure storage files in local firstly and then load them into a SQL DB.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: tAzureStorageGet directly into tFileInputDelimited

Thanks Sabrina.  I have limited TEMP storage so would need to iterate through the items in Azure, download each locally, process, remove and repeat.  Could you give me a hint as to how to achieve that please?  I'm not sure how to use 'AzureStorageGetList" as an input into "AureStorageGet"

Moderator

Re: tAzureStorageGet directly into tFileInputDelimited

Hello,

Here is a scenario about:TalendHelpCenter:Retrieving files from a Azure Storage container. Hope it will help.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog