How to log tRestClient responses into their respective input field records

Six Stars

How to log tRestClient responses into their respective input field records

Hi,

  I've created a job that returns JSON records from a GET REST endpoint, from 'system A', and map all fields, including an 'system A' ID for each record from that system, and use tWriteJSONFields to convert into another JSON schema format which is Posted to another REST endpoint, of another 'system B', which returns a 'system B' ID's, for the newly created records.

How do I match up the'system 'A' input data + 'system 'A' ID with the response data from 'System B' + 'System B' ID into a CSV file.

 

I tried assigning the 'System A' ID to a context var in tJavaRow, but the tRestClient uses the last stored 'System A' ID from the last assigned context var from the last iteration of the flow?

 

Screen Shot 2018-09-14 at 5.39.12 PM.pngCheers,

 

Synfield.

Community Manager

Re: How to log tRestClient responses into their respective input field records

I'm not sure precisely what you are asking, but I *think* I kind of get the rough idea. I think you are asking about a way of ensuring you can take the data from one Rest service response, store it, send it to a second Rest service and link the first response record with the record returned by the second Rest service. Is that about right?

 

If so, there is an easy way of doing this. What you need to do is keep your job the same up until the record is returned from the first Rest service. Then use a tFlowToIterate which will allow you to, for every record returned from the first Rest service, carry out the logic for the second Rest service and then join the data. The reason for this is that the tFlowToIterate essentially pauses the preceding components until the rest of the flow is complete. So your records stored (in the globalMap by the tFlowToIterate) for the first Rest service are kept static for each iteration. There is an iteration of everything after the tFlowToIterate for every record returned by the first Rest service.

 

 

Six Stars

Re: How to log tRestClient responses into their respective input field records

Thanks for this rhall_2_0. Your assumption is correct, I'm basically taking records from one system endpoint and mapping fields to post into another system via their respective endpoints

tRestClient ----- GET JSON response ---- tExtractJSON ----tMap ------(insert response fields into DB) --------tWriteJSON------

tRestClient-----POST JSON ---- tExtractJSON (response I'd of new record insert into DB match with records from lst tRest response)

 

Could you please give me a better understanding of where I would place tFlowToIterate in the above data flow.

Thanks for your help.

Community Manager

Re: How to log tRestClient responses into their respective input field records

Looking at your flow I would suggest placing the tFlowToIterate after your tExtractJSON (after the first tRestClient). As a slight change to what I suggested before, add a tFixedFlowInput via Iterate link after the tFlowToIterate. Your column data will be added to the globalMap. You will need to retrieve this data and output it in the tFixedFlowInput component. To do this, copy the schema of the tExtractJSON schema (output schema) and paste it in the tFixedFlowInput. Then you will need to retrieve the data for each column from the globalMap. To do that, you need to be aware of the row name, the column names and the types (classes). For example, if you have a column called "name" which is a String and the row feeding the tFlowToIterate is called "row21", the globalMap code you would use to retrieve this value for the tFixefFlowInput "name" column would be.....

 

((String)globalMap.get("row21.name"))

In the following flow, you can use the above syntax to retrieve the values stored in the globalMap via the tFlowToIterate. 

Six Stars

Re: How to log tRestClient responses into their respective input field records

I think I understand what your suggesting which is to store all the fields, on each iteration, into a globalMap, but I'm already storing these as rows in a DB, after tExtractJSON, from lst tRestClient, but I'm having trouble with updating DB, with the response ID's from the second tRestClient, as both Id's are unique to the system that the same record belongs to, I.e I'm transferring employee records from one system and creating them in another.

How would I then match the responses from the second tRestClient responses (newly created ids) and update their respective records in the DB, with the new id, that where inserted after the lst tRestClient?

Hope this makes more sense. 

 

Cheers.

 

Community Manager

Re: How to log tRestClient responses into their respective input field records

You need a common ID to do this. I thought your issue was that your common ID was coming from your first Rest call and that you couldn't use it after your second Rest call. Sorry, my mistake there. 

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Downloads and Trials

Test drive Talend's enterprise products.

Downloads