[resolved] tSetProxy with tRest

Six Stars

[resolved] tSetProxy with tRest

Hi All,
This is my first attempt to use Talend to obtain data from an API and i am having some trouble getting it to work...
So far i have a very simple job that sets proxy settings (tSetProxy) and once this completes it goes to a tRest that connects to the API using a https address with authentication passed to it in a http header.
I have used tSetProxy successfully when access files on an ftp server so i am confident that the settings against this are correct.
However, i still get the following error when it hits the tRest:
Exception in component tREST_1
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection timed out: connect
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:131)
at com.sun.jersey.api.client.Client.handle(Client.java:616)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:559)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:72)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:454)
at microstrategy.zendesktest_0_1.ZenDeskTest.tREST_1Process(ZenDeskTest.java:718)
at microstrategy.zendesktest_0_1.ZenDeskTest.tSetProxy_1Process(ZenDeskTest.java:560)
at microstrategy.zendesktest_0_1.ZenDeskTest.runJobInTOS(ZenDeskTest.java:1086)
at microstrategy.zendesktest_0_1.ZenDeskTest.main(ZenDeskTest.java:832)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.security.ssl.SSLSocketImpl.connect(Unknown Source)
at sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.<init>(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.New(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:218)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:129)
... 8 more

Which suggests that either the API didn't respond or there is a proxy issue.
Here's the fun part - if i run netstat while this job is running i see the following record:
  TCP    10.114.2.6:51867       185.12.82.1:443        SYN_SENT    

Well, this suggests that tRest is trying to access the API directly and not through the proxy - so it's ignoring the details given to it by the tsetproxy command...  I don't see any settings in tRest to force it to use the proxy settings so i am unsure how to proceed with this now.  I could also use a tJson component, but i cant see any way to set the authentication details in this one.
I'm a relative beginner when it comes to TOS so please be kind :-)
Running Talend Open Studio for Data Integration v6.0.0, Build id: M4.
Regards
Scott
Six Stars

Re: [resolved] tSetProxy with tRest

so...
I swapped out tRest with tRESTClient and i now have proxy options in this component.
However, not i get an error:
Exception in component tRESTClient_1
javax.ws.rs.NotAuthorizedException: HTTP 401 Unauthorized
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.cxf.jaxrs.client.AbstractClient.convertToWebApplicationException(AbstractClient.java:483)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:897)
at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:865)
at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:428)
at org.apache.cxf.jaxrs.client.WebClient.get(WebClient.java:611)
at microstrategy.zendesktest_0_1.ZenDeskTest.tRESTClient_1Process(ZenDeskTest.java:673)
at microstrategy.zendesktest_0_1.ZenDeskTest.runJobInTOS(ZenDeskTest.java:1045)
at microstrategy.zendesktest_0_1.ZenDeskTest.main(ZenDeskTest.java:791)

Which suggests the username and password are incorrect in the HTTP header - but testing the same values in PHP returns data without any issues...
Any suggests?
Six Stars

Re: [resolved] tSetProxy with tRest

OK all sorted - has the Authorization values in the wrong place!!!
One Star

Re: [resolved] tSetProxy with tRest

Hi Scott,
Can you elaborate please? Which one was the wrong place and which one is the right place? I am having the same problem. I am using tRestClient. Had the authorization code as query parameter and also as the header in Advanced settings. Both returned NotAuthorizedException error. 
Thanks
Arvind.
One Star

Re: [resolved] tSetProxy with tRest

Originally i was using tSetProxy before the tRest, but tRest didn't seem to picking up the tSetProxy details.
However, on the advanced tab of tRest there is a section where you can add proxy details.
Once i added the details here, i no longer had issues, and no longer needed the tSetProxy.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Download