Issue with JSON generated by TDynamoDBInput

Highlighted
Four Stars

Issue with JSON generated by TDynamoDBInput

Hi, thanks in advance for your support and responses

 

We are using the TDynamoDBInput component to connect to a Dynamo DB table and dump the records.

The JSON which gets dumped is of the below format (Please note this is indicative and not actual values)

 

{key1 = { key2=value2 , key3=value3}}

 

Now the issue faced by us is that this is not identified as a valid JSON by Redshift Spectrum or any JSON parsing tool

since an ideal JSON would be

{"key1" : { "key2":"value2" , "key3":"value3"}}

 

( Note the double quotes / Colon which are required in a well structured JSON)

 

Are we doing something incorrect. Can you please guide.

As such the generated JSON by DynamoDB input cannot be used by any tool which parses JSON.

 

Thnaks

 

Highlighted
Five Stars

Re: Issue with JSON generated by TDynamoDBInput

I just fought with this same issue.  DynamoDB uses what it calls DynamoDB JSON, which is not formatted like a classic JSON string.  What I ended up doing was using the tReplace connector to reformat the JSON string into something usable and then used tExtractJSONFields to break down the JSON string into it's usable parts.

Highlighted
Six Stars

Re: Issue with JSON generated by TDynamoDBInput

Hi all, 

unfortunately I'm facing the same problem, tested with V7.3.1.M3.

This bug is still not fixed.

 

@xdshi Can you please add it to your bugtracker and suggest it for implementing the correct quotings within the final V7.3.1 version?

 

Many thanks in advance!

Highlighted
Six Stars

Re: Issue with JSON generated by TDynamoDBInput

@shong  @rhall_2_0  

Do you have any updates, if it will be fixed for final V7.3.1, I hope so?

In V7.3.1M6 its still the old problem and would not be usable!

 

Kind regards!

 

 

Highlighted
Community Manager

Re: Issue with JSON generated by TDynamoDBInput

@Xenoflex, the milestone version is just for testing, please wait for the GA release 7.3.1 which will be available in the second half of this month. If the problem is not fixed yet, please open an issue with details on our bugtracker?

Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Community Manager

Re: Issue with JSON generated by TDynamoDBInput

Has anybody raised this as a bug? Bugs can be raised and searched for here https://jira.talendforge.org/

 

If this is caused by DynamoDB JSON, I am not sure R&D will accept this as a bug, more of a feature request. If you are getting data back in the format pointed out in the first post (unfortunately I cannot test this as I don't have access to a DynamoDB instance at the moment), this should be relatively easy to correct using some simple String manipulation. Perhaps @pclay can share their method of doing?

 

In the meantime, feel free to raise this feature request. However, if it hasn't been raised previously there will be no change in the 7.3.1 release. Raising an issue on the Community is not the same as raising an issue in Jira.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now