Downloading files of a sub-folder in azure storage container is saving folders as empty text files

[sudheerjami]

Hi,

I am using TOS_BD_V6.4.1 on Ubuntu. I am trying to download files from a azure storage container using tAzureStorageGet. My folder structure in azure storage is "dev/TestFolder/inside/abc.xlsx" (dev is the container) and I am trying to download all the files in "TestFolder" directory to a local folder "Test" in downloads directory.

But when I run the package, it is creating a file "TestFolder" inside "Test" directory in my local and then throws "/*/*/inside Not a directory" error.

Below is my tAzureStorageGet config:

Below is the error:

Please help and thanks in advance.

[ArvinRapt · Accepted solution]

You have to use the global variable of the tAzureStorageList component.

Can you try giving this value - "((String)globalMap.get("tAzureStorageList_1_ACCOUNT_KEY"))" in the prefix and try again?

[ArvinRapt]

I think in most cloud storage technologies, there is no concept of a 'sub-folder' which is created explicitly.

For example, the name of the file abc.xlsx is actually "TestFolder/inside/abc.xlsx".

So in your component you can download only one file at a time. What I would suggest is to use the tAzureStorageList Component to get all the files names under the TestFolder prefix and connect the iterate component to the tAzureStorageGet Component to download the files.

[sudheerjami]

Hi ArvinRapt, thanks for your response.

Earlier, I was following the example that was mentioned here. Even when I use iterate to download the files list as you mentioned, i'm getting the same error. But on iterate over the list i can see the names of all the files and folders in the console using tJava.

[ArvinRapt]

can you post a screenshot of the tAzureStorageGet component and how its connected to the iterate link?

That'll be more helpful.

[sudheerjami]

ArvinRapt, thanks for your time.

Below is my tAzureStorageGet component

Here is the flow:

I noticed that if i put the entire file path of the file in prefix ("TestFolder/inside/abc.xlsx") it is downloading that file but i want to download all the files in the directory.

[ArvinRapt · Accepted solution]

You have to use the global variable of the tAzureStorageList component.

Can you try giving this value - "((String)globalMap.get("tAzureStorageList_1_ACCOUNT_KEY"))" in the prefix and try again?

[sudheerjami]

ArvinRapt,

I am getting "Can't open the specified file input stream from file" error on using the global variable of the tAzureStorageList component as shown below.

[sudheerjami]

ArvinRapt,

I removed the test folder and tried with a new folder and it worked. But it is downloading all the files under dev container (other folders as well). Can you please let me know how i can limit it to download from one folder.

[ArvinRapt]

Using the tAzureStorageList component? or by just using the tAzureGet Component?

Either way I would suggest checking the value that you are passing to the prefix parameter.

[sudheerjami]

ArvinRapt,

I used ((String)globalMap.get("tAzureStorageList_1_CURRENT_BLOB")) in tAzureStorageGet as prefix and iterated on the tAzureStorageList_1 list. It solved my issue.

Thanks a lot for guiding me through.

[ArvinRapt]

That's great. You are welcome.