Moving files in Azure

Six Stars

Moving files in Azure

I need to process CSV files stored in Azure in a SQL database.  I know I can't read the files directly out of Azure, so I'm downloading locally and then processing.  Once processed, however, I need to move the processed files form one Azure container to another.  This is where I'm having an issue.  How do I "move" data  (with sub folder, which I know in Azure is really just the file name) from one container to another?


 To be more specific:

Get Temp folder name -> tAzureStorageList of source container --iterate-> tAzureStorageGet  -> tFileList -> [process into SQL] -> tAzureStoragePut (file name produced by tFileList)


The throws an error: [ERROR]: org.talend.components.azurestorage.utils.AzureStorageUtils - No match file

tAzureStoragePut Local folder = temp path

tAzureStoragePut Filemask = ((String)globalMap.get("tFileList_CURRENT_FILE_PATH"))azu



Seven Stars

Re: Moving files in Azure

Hi beerygaz,


In the file mask instead of  ((String)globalMap.get("tFileList_CURRENT_FILE_PATH")) use ((String)globalMap.get("tFileList_1_CURRENT_FILE")).

Thanks & Regards
A Ravi Kumar
Mobile Number : +91 852-762-1083
Email-id :
Skype Id : ammanannaravikumar
Six Stars

Re: Moving files in Azure

Hi Ravi, I get a similar issue:


CURRENT_FILEPATH: C:\Users\gavin\AppData\Local\Temp\BIData\Station A\2017_10_01.csv

CURRENT_FILE: 2017_10_01.csv


Both produce: 

[ERROR]: org.talend.components.azurestorage.utils.AzureStorageUtils - No match file "C:\Users\gavin\AppData\Local\Temp\BIData\Station A\2017_10_01.csv" exist!

[ERROR]: org.talend.components.azurestorage.utils.AzureStorageUtils - No match file "2017_10_01.csv" exist!


-- Edit -- 

If I remove the "C:\Users\gavin\AppData\Local\Temp\BIData\" with a .replace function, I am left with:

Station A\2017_10_01.csv - but the tAzureStoragePut still only puts the file in the root of the container, it does not create the folder "Station A" 

And tAzureStorageDelete will remove files in the root, but not files within a folder.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.