Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

One Star

Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

Hi Team,
I'm using the BigData Open studio for BigData 6.3.1
My Hadoop Cluster is up and running with Cloudera Manager 5.10.0 managing it. I'm trying to create a metadata connection to the cluster by retrieving configurations from Cloudera Manager. 
I'm able to connect to the Cloudera manager from the Talend Metadata Import Wizard, and it fetches the services running on it. In the next step where we have to configure name node and configuration manager URI, I enter the details and check the connection again.
The namenode connection is successful but the configuartion manager connection isn't. This is the error report generated:-

org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:55)
at org.talend.designer.hdfsbrowse.hadoop.service.HadoopServiceBean.check(HadoopServiceBean.java:102)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckHadoopServicesDialog$5.run(CheckHadoopServicesDialog.java:373)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:47)
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:52)
... 5 more
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:44)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.talend.core.utils.ReflectionUtils.newInstance(ReflectionUtils.java:278)
at org.talend.designer.hdfsbrowse.hadoop.service.check.provider.CheckedResourceManagerProvider.check(CheckedResourceManagerProvider.java:51)
at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider$1.run(AbstractCheckedServiceProvider.java:47)
at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit$1.call(CheckedWorkUnit.java:65)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 more
Caused by: java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
at org.apache.hadoop.mapred.JobClient.init(JobClient.java:472)
at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:450)
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px '.SF NS Text'} span.Apple-tab-span {white-spaceSmiley Tonguere}
... 12 more

It's possible that the cluster initialization is failing because I'm telling the Talend wizard that my Cloudera Manager version is 5.8 YARN (It's actually 5.10.0). The jars for 5.8 may not be compatible with 5.10. Is there any way to get the right Cloudera Manager option in the wizard?
One Star

Re: Hadoop Cluster Configuration: Cloudera Manager version 5.10.0

Found the solution,
Had to add the IPs and the hostnames to my Mac's /etc/hosts. Talend wizard was then able to resolve the DNS
P.S. You can configure Cloudera Manager 5.8 YARN in the Talend Metadata wizard if you want to use 5.9 and above. It works!

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now