handling of schema dynamically

Six Stars

handling of schema dynamically

hello all..

i need to convert xml file to json and insert that into maprdb. How can I handle schema dynamically in that.

Suppose my XML has 10 fields and most of the time  3 ,4  field contains null values so in that case i just want to store rest fields having values  and i don't want to store all 10 fields.But in some time all the fields have values and there is need to store all the fields.

So to avoid wastage of space i just want to save the fields having some value other then null.Can in do this in talend? 

Ten Stars

Re: handling of schema dynamically



There's a component to convert from XML to JSON



Sidenote, basics Document type which you easily can convert to json i assumen... 


Good Luck

Six Stars

Re: handling of schema dynamically


I am able to change xml to json that is not the problem for me.

I want to handle schema dynamically. Like if 4 columns have value that 4 columns should be stored in maprdb and if 7 columns have values than 7 columns should be stored.


Re: handling of schema dynamically

@shalla - not entirely sure on your use case. But I can think of an approach: If your input would have been a delimited , the solution would hav ebeen lil different For XML ------ Option 1 pick up the maximum possibility hierarchial xml schema file and use as input. tXMLInput (xpath is just "/" -> tXMLMap -> Map all fields to right -> use tJavarow like to filter out or write conditions -> flatten out the file -> arrange back to JSON. Options 2 I have never tried data type as Dynamic with XML type , give it a try. For delimited dynamic data type schema works perfectly fine. only available in enterprise not TOS.
Six Stars

Re: handling of schema dynamically

@snayak thanku for your response . I will try try that.I am using enterprise.

My use case is dynamically handle the schema in terms of number of columns.Some time xml file has all 30 fields and some time 30-N .


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences


Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now