JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Four Stars

JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Hello,

I have a "tREST" component that I link to a tJSONoutput component
In the tREST I set up a cURL request, the request requests a URL and an "X-Auth-Token" in HTTP header
After that I get a JSON that contains all the metrics of my infrastructures on openstack

the concern is that the json is not the right format, it's a big package of unstructured data in JSON format.

The solution is to add "| python -mjson.tool" at the end of the cURL request.

Like this: curl -H "X-Auth-Token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX" "XXXXXXXXXXXXXXXXXXXXXX / servers / detail" | python -mjson.tool

This works very well on the bash github console, but I would like to use it on TALEND so that my tREST interopts openstack and generates a JSON with the GOOD FORMAT.

Thank you for your help ...

 

ahmed.daoudi.pro@gmail.com

Fifteen Stars TRF
Fifteen Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Hi,

Essayez de reformuler en anglais pour augmenter vos chances d'obtenir une réponse (les gens sont principalement anglophones sur ce forum). 


TRF
Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Hi, my message was in English, it had to translate automatically because I have an automatic translator on my internet pages. Thank you
Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Can you explain what you mean by a good format? What is being sent back by the tRest that you do not like? 

Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

By "good format" I mean a JSON "normal" with indentations, returns to the line. There with my job TALEND I get a big packet of data, a big block.

MY JSON :

 

{"data":[{"Body":"{\"servers\": [{\"status\": \"ACTIVE\", \"updated\": \"2018-01-23T09:45:02Z\", \"hostId\": \"XXXXXXXXXXXXXXXXXXXXXXX\", \"addresses\": {\"reseau_1\": [{\"XXXXX:mac_addr\": \"XXXXXXXX\", \"version\": 4, \"addr\": \"XXXX\",

good format wich I want:

{

    "servers": [

        {

            "status": "ACTIVE",

            "updated": "2018-01-23T09:45:02Z",

            "hostId": "XXXXXXXXXXXXXXXXXXXXXXXXXX",

            "addresses": {

                "reseau_1": [

                    {

                        "XXXXXXX:mac_addr": "XX:XX:XX:XX:XX:XX",

                        "version": 4,

                        "addr": "XX.XX.XX.XX",

                        "OS-EXT-IPS:type": "fixed"

 

Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Ah I see. This is relatively easy, but requires a little bit of code. Output your JSON to a tJavaFlex. Assuming the row that enters the tJavaFlex is called "row1", the row that exits the tJavaFlex is called "row2" and the column for the JSON input/output is called "json", this code should work for you....

 

JSONObject json = new JSONObject(row1.json); 
row2.json = json.toString(4); 
Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

ok i will try to do it and tell you what it is. So it's impossible to do as in curl with GIT BASH, just adding "| python -mjson.tool"? It works very well in bash I receive a JSON file in the right format. It would be so much easier to add "| python -mjson.tool" somewhere in talend ... Thank you for your return!

Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Unfortunately you are comparing Talend to a number of different technologies working together there. You have BASH, Curl and Python working together. I guess you are more familiar with Python than Java, which is why it seems easier. But adding a couple of lines of Java is really very simple and one of the massive advantages of using Talend. Although granted, it does help if you know Java :-)

Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

your code does not work at home. I think you did not understand what I want then. i have a component tREST that will type on an openstack base in cURL. this component is connected to a tFileOutputJson component, the tFileOutputJson writes all the metrics of my openstack infrastructure in a json file. In this json file I have all the information and metrics I want. But they are not in JSON format, they are poorly structured. With GIT BASH it works fine, I send a cURL request and receive the JSON information, but I would like to do this on TALEND in order to continue and automate my jobs. In addition, with GIT BASH you can not generate a JSON file, it's just the display. As I told you, with GIT BASH just add "| python -mjson.tool" at the end of the request curl and it's very simple. But I do not know where to put the "| python -mjson.tool" with talend. Thanks for your help.

Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Is it possible to copy / paste this request cURL in TALEND? curl -H "X-Auth-Token: xxxxxxxxxxxxxxxxxxx" "https: //compute.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/servers/detail" | python -mjson.tool

Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Change the  tFileOutputJson component with a tFileOutputRaw component. Then do not open the file with Notepad. This works.

Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

You can do practically whatever you want in Talend. However doing what you're wanting to do is a little like putting a road inside a train. 

Four Stars

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

thank you ! but your answer does not completely solve my problem! I used a tFileOutputRaw but indentation and line breaks were not taken into account. So I can't parser my json to extract information! help me please

Community Manager

Re: JSON issue not compliant output (Curl - JSON - tREST - TALEND)

Indentation and line breaks should not affect a true parser. Anyway, if you open the file using an application that does not ignore formatting (something like WordPad in Windows), you will see that it is formatted in a more readable fashion. 

 

This is not a Talend issue, this is an interpretation issue.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now