Five Stars

Batch JsonInput from Folder

Hi,

 

I am trying to extract some data in CSV format from around 40,000 Json files.

Assuming that all files contain the same fields, what would be the best/most effecient way to process these?

 

Thanks

K

 

1 ACCEPTED SOLUTION

Accepted Solutions
Forteen Stars TRF
Forteen Stars

Re: Batch JsonInput from Folder

Hi,

The genral design should look like this:

tFileList-->tFileInputJSON-->(tSomethingElseIfNeeded)-->tFileOutpoutDelimited

Click on the "iterate" link after tFileList then tick the option"Enable parallel execution" then enter the number of threads you want in parallel (maybe something like 4 or 5).


TRF
2 REPLIES
Forteen Stars TRF
Forteen Stars

Re: Batch JsonInput from Folder

Hi,

The genral design should look like this:

tFileList-->tFileInputJSON-->(tSomethingElseIfNeeded)-->tFileOutpoutDelimited

Click on the "iterate" link after tFileList then tick the option"Enable parallel execution" then enter the number of threads you want in parallel (maybe something like 4 or 5).


TRF
Five Stars

Re: Batch JsonInput from Folder

You are brilliant, thank you.
it is worth noting that in the filename path for the jsoninput you must specify the variable from tFileList_1:

((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))