Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

One Star

Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi, 

I am running a job on Amazon EMR for tHDFSPut component, while running the Job i'm getting org.apache.hadoop.ipc.Clent - Retrying connect to server. I have provided with the screenshot. Can anyone shed some light on this part and provide me with the solution?

Click here for the screenshots
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi 
Not able to see your screenshot,please upload it again so that we can help you.

Thanks
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi Sanjay,

I have updated the post please find the link to get the screenshots.

Thanks.
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi 
In your screenshot,i am still not able to see your component setting.Seems have some problem with configuration you have done in tHDFSPut component.Also make sure you are using correct IP and port and Hadoop is up and running at that port at EMR.please verify that from configuration file of Hadoop.
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Thanks for the response Sanjay.
I have put the configuration for the component in the second image in the screenshot. I have put a screenshot (click to open) again for your reference. 
Basically, I am just trying to test Hadoop connectivity with a simple file transfer by using the tHDFSPut component. I have verified that Hadoop cluster is up and running. I got the port number from the core-site.xml file in /home/hadoop/conf folder in the master node of the EMR cluster. 
  <property><name>fs.default.name</name><value>hdfs://10.0.0.67:9000</value></property>
I have also tried creating a SOCKS5 proxy tunnel, as mentioned in the AWS documentation, from the Talend machine to the EMR master node.
I am not sure if I am missing some step. Very much appreciate any help.

Thanks
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi
Your configuration is correct,then it must be some issue related to your /etc/hosts file on EMR.Your master node is continuously listening to specified IP and it is not accepting any other hosts.So do one thins
1) Open sudo gedit/etc/hosts
2) Add 0.0.0.0 hostname(Initially it is like 127.0.1.1  hostname).
Hope it will help you.
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hi, 
I tried the same , here is the conetent of hosts file. but still i'm getting org.apache.hadoop.ipc.Clent - Retrying connect to server.
Really appreciate if anyone could help me on this.
#127.0.0.1      localhost
::1             localhost ip6-localhost ip6-loopback
fe00::0         ip6-localnet
ff00::0         ip6-mcastprefix
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters
0.0.0.0  localhost
10.0.0.184 ip-10-0-0-184.us-west-2.compute.internal
0.0.0.0  localhost

Thanks.
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

Hey 
your your /etc/hosts file is configured for a single machine,it will not work in networking mode.You need to use FQDN(Fully qualified domain names).
It should be like:
     IP                         FQDN                    Alias
127.0.0.1       localhost.localdomain       localhost

You can follow this link for more detail

http://linux.about.com/od/lna_guide/a/gdelna46.htm

Hope you get to the solution.
Four Stars

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

I'm having the same issue and above suggestions didn't work.
One Star

Re: Amazon EMR org.apache.hadoop.ipc.Clent - Retrying connect to server

I created the EMR in a VPC connected with the local IP and it run's (must have access to the internet during startup of the cluster)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog