Parsing multiple input JSON files

One Star

Parsing multiple input JSON files

Hello All,
We have a requirement to read data from multiple json files stored at a particular location and transform all data and finally insert it to a postgre db. 
I am using Talend tFileInputJSON component to read json file, but i can read only 1 JSON file. 
Since i am fairly new to Talend, looking for experts opinion here on how to tackle this situation. 
Any support appriciated.
Thanks,
Manesh
Five Stars

Re: Parsing multiple input JSON files

Welcome to Talend! 
follow the instructions to get your job done. 
Your job design should be like this. 
tFileList--iterate--tFileinputJson---tMap--youroutput.

Add tfileList component, set the directory path where all files stored. 
Add file mask as per your file names default with "*.*". 
Connect tFileList to tFileInputJson with "iterate" flow. 
you will have " file name with path" in tFileList global variables. add this global variable to tFileInputJosn file path text box. 

global variable which store the file name with path. 
((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
One Star

Re: Parsing multiple input JSON files

Thanks for your help Umesh.. Multiple file parsing Worked perfectly !
One question, can tMap hold data from all the json files before inserting into db or do i need to insert one by one ?
Thanks,
Manesh
Five Stars

Re: Parsing multiple input JSON files

use tBufferoutput and input component to hold the result in buffer then insert this result in db. 
One Star

Re: Parsing multiple input JSON files

It's often easier to convert the JSON to a CSV flat file first then import that into tables.
There are some free converters like json-csv.com
One Star

Re: Parsing multiple input JSON files

Thanks Umesh and Holga for your inputs. I will try to follow it. Basically my JSON has few nested loops and i need to put some custom logic before inserting in database,so converting into csv may not help my purpose. 
Five Stars

Re: Parsing multiple input JSON files

if you share input json file, then we can suggest better solution. 
One Star

Re: Parsing multiple input JSON files

Thanks Umesh,

Below is my sample json file with very few fields, 
{"prod_code":"2568945","default_sku_cd":"010101001","sku_list":,"color_details_list":,"size_details_list":,"length_details_list":,"price":"12.00",}

So here I want to achieve 2 things,
1st Job : read each file like above and Insert sku_list row by row into db, but before that i need to map fields like color_code in color_details_list array to get respective color name or if any field is null then replace by "-" before inserting record. 
2nd Job : Insert level 1 fields (ex. prod_code,default_sku_cd,price etc) but before that loop through sku_list  for few details (ex. sku_price,color_code,color_name) for sku_code matching with default_sku_cd and then add all these details as  a single row. 
Since i am fairly new to Talend, unable to decide what approach to take to find solution for above problems. Any directions, help is highly appreciated.