Six Stars

modify existing JSON schema - add new column and supply JSON path for new column

hello everyone

 

I am using Talend DI 6.4.1

 

I have JSON metadata created from input JSON file.

 

Now I need to add 2 new columns to that existing input file schema and I need specify the JSON path for them all.

I can add the new columns easily to the schema but I can not find where I can provide the JSON path for the new columns in the existing schema

Please advise.

THANKS!

 

  • Data Integration
Tags (1)
1 REPLY
Six Stars

Re: modify existing JSON schema - add new column and supply JSON path for new column

I fixed it.

 

It looks like a bug in Talend DI GUI/UX re XML file generation for JSON metadata when making schema changes such as adding new columns to existing JSON input file.

 

I edited the input JSON file for new data so to add new columns and then updated the metadata in DI UI.

This step only did half-job which is why I was not able to see my new JSON columns flowing through.

The XML file generated by Talend DI was missing the new columns, so I edited the metadata XML file (.item) in Notepad++ and now it is all working.

 

This is the code that UI did *not* generate for new columns EventArg and EventSource which I had to add manually to the XML file (<schemaname>.item)

 

<feature xsi:type="TalendMetadata:MetadataColumn" xmi:id="_-QVmlXaIEee5aJ1ewsGKDw" label="EventArg" length="100" talendType="id_String" pattern="&quot;dd-MM-yyyy&quot;"/>
<feature xsi:type="TalendMetadata:MetadataColumn" xmi:id="_-QVmlXaIEee5aJ1ewsGKDw" label="EventSource" length="100" talendType="id_String" pattern="&quot;dd-MM-yyyy&quot;"/>

 

Given this code was generated by the DI:

<schemaTargets xmi:id="_9EpyUHaPEeeiW_0TZlhfbA" RelativeXPathQuery="_source.query.EventArg" TagName="EventArg"/>
<schemaTargets xmi:id="_D7mcAHaQEeeiW_0TZlhfbA" RelativeXPathQuery="_source.query.EventSource" TagName="EventSource"/>

 

Can anyone tell me about the xmi:id="_9EpyUHaPEeeiW_0TZlhfbA" please?

The "id" value seems to be some sort of unique random number like a GUID.

How does Talend ID generate this number and what is it used for internally?

 

I am asking because if I can generate this "id" myself, it is actually less work for me to manually the XML file instead of using Talend DI UI to try to maintain the JSON schema changes. The DI UI has major restrictions - for example, if my JSON input file is ~10-15 MB in size, the UI simply chokes when I try to create metadata or it tells me that JSON format is incorrect when in fact it is perfectly correct and at run-time Talend Job has no issues processing the same file.

 

Thanks!