tREST http body problems using global variable

Seven Stars

tREST http body problems using global variable

Hello everyone

 

I have looked at previous questions on this topic but I suspect I may have a different issue with a global variable I am trying to pass into tREST http Body.

 

I am definitely doing something wrong in my Talend DI job.

 

The following fails:

 

http body in tREST calling ElasticSearch set like so:

 

((String)globalMap.get("myscrollid"))

 

Error from ElasticSearch results:

 

{"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId];

nested: IOException[Bad Base64 input character decimal 46 in array position 4]; ","status":400}~400

 

When I use the hard-coded value of the ScrollID in http Body and surrounded by double-quotes like so:

http Body:

"c2NhbjsxMDsxODQ1OTM5ODE5OndCZktsWUo1VHYta3RvdWZiUmMzWFE7MTg0NTkzOTgyMDp3QmZLbFlKNVR2LWt0b3VmYlJjM1hROzE4NDYyNzg2ODk6bkE2T0ZVVWtRcFdNMktLT0tmaTRUBBISHMjc4NjkwOm5BNk9GVVVrUXBXTTJLS09LZmk4M1E7MTg0NDQ0NjcyNzo4cE9xcjB0S1NfQ195M0VmdWpSMlFnOzE4NDQ0NDY3Mjg6OHBPcXIwdEtTX0NfeTNFZnVqUjJRZzsxODQ0MTA3MzM4OkZIdnkyaXEzUXMtSS1RR0htRTlnRlE7MTg0NDMwNjE1MTpTelRndzJmN1J2Q0JTWlRNdUpRaVNBOzE4NDU2MDkyOTI6Q0xzTllsVl9TdmF5em9RVW1lOWhfdzsxODQ0ODkzMDg5OjJTTkpRNUd0U1MyWlg3ekNQWmFMd0E7MTt0b3RhbF9oaXRzOjEzNzIwNTs="

 

Then it works.

 

So I know the software on client and server is OK and it must be my error in Talend DI job. 

 

I am doing something wrong passing the value stored in global variable to http body.

 

I have verified the value of global variable by using tLog and writing the value to csv file, so I am sure the value itself is OK, since I used it as hard-coded value to prove it works.

 

Somehow the way I extract that value programmatically into http body upsets ElasticSearch ! 

 

Thanks for any tips and clues!

 

 


Accepted Solutions
Seven Stars

Re: tREST http body problems using global variable

I think I solved it, but need to confirm with you.

 

in a Key - Value for the global variable I had this in Value:       "row5.scroll_id"      - i.e. surrounded by double quotes "".

Because it is expression I had to remove those double quotes like so:         row5.scroll_id 

 

It seems to be working now, but please confirm this for me so others may also benefit from my Learner mistakes ! 8^) LOL

I was passing scroll_id string value of "row5.scroll_id"  to the http body of tREST, it seems.

 

Community Manager

Re: tREST http body problems using global variable

I think I see. So you went from this....

globalMap.put("gbl_varname", "row5.scrollid") 

....to this....

globalMap.put("gbl_varname", row5.scrollid) 

If so, that explains everything. Putting quotes around text makes it a String Literal in Java.


All Replies
Community Manager

Re: tREST http body problems using global variable

I suspect that this may be relatively straight forward, but we will need more info to give you an answer. Can you post a couple of screenshots showing your job, where you set the globalMap variable and you tRest config please?

Seven Stars

Re: tREST http body problems using global variable

I think I solved it, but need to confirm with you.

 

in a Key - Value for the global variable I had this in Value:       "row5.scroll_id"      - i.e. surrounded by double quotes "".

Because it is expression I had to remove those double quotes like so:         row5.scroll_id 

 

It seems to be working now, but please confirm this for me so others may also benefit from my Learner mistakes ! 8^) LOL

I was passing scroll_id string value of "row5.scroll_id"  to the http body of tREST, it seems.

 

Community Manager

Re: tREST http body problems using global variable

I'm not sure what you have done here. Are you still referring to a globalMap variable or are you just referring to the row value? Can you take a screenshot please?

Seven Stars

Re: tREST http body problems using global variable

in designer in Key-Value
I have "gbl_varname" in Key (in double-quotes) and "row5.scrollid" in Value in double-quotes.
I should have had rows5.scrollid in Value *without* double-quotes
Community Manager

Re: tREST http body problems using global variable

I think I see. So you went from this....

globalMap.put("gbl_varname", "row5.scrollid") 

....to this....

globalMap.put("gbl_varname", row5.scrollid) 

If so, that explains everything. Putting quotes around text makes it a String Literal in Java.

Seven Stars

Re: tREST http body problems using global variable

yes, exactly, and I did not see this until I examined the generated Java code

 

which means that even if you do not write any Java code to be able to work with Talend you still need to be able to read and understand Java code.

 

 

 

Community Manager

Re: tREST http body problems using global variable

Yes, you do need Java.....which is no bad thing in my opinion. While some people wishing to sell development products like to be able to claim "my grandmother can use this", it really helps nobody when you end up with a "development team" who do not know how many bits are in a byte :-) 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog