Hadoop Cluster connection to Amazon EMR

Employee

Hadoop Cluster connection to Amazon EMR

Hi,
Has anyone managed to connect successfully to Amazon EMR (apache version 2.4.0) from local open studio for Big Data 5.6.0 install?
I understand you have to create a ssh tunnel config to enable communication to Amazon EMR.
This ssh connection is successfull as the localhost routed can display the name node as below screenshot:
 



The error occurs when the check services button is clicked.
See below:



The error logs are as below:

                                                                           
 



Any help here much appreciated.

Thanks,
Employee

Re: Hadoop Cluster connection to Amazon EMR

Hello,
It seems like there is a problem to dispay the EMR error message. But I think I have the solution to your problem.
On amazon 2.4.0, the namenode URI has the port 9000, and the resource manager 9022.
Could you try with the following settings:
Namenode: hdfs://localhost:9000
Resource Manage: localhost:9022
if it does not work, could you try with the full URL (don't use the internal one, use the one starting with ec2-XXX):
Namenode: hdfs://ec2-xxx.amazonaws.com:9000
Resource Manage: ec2-xxx.amazonaws.com:9022

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now