One Star

Establishing connection to Hadoop Cluster Hortonworks V2.2.3

Hello,
I'm working with Talend Big Data Platform (V 6.2.1) and I'm having trouble to establish a connection to our HDFS. I created an Hadoop Cluster in Metadata and imported all settings directly from ambari. I have to use a Kerberos authentication with a keytab which I already tested successfully with Hortonworks ODBC Driver for Apachi Hive (V2.1.2).
Checking the connection I receive some errors mostly about illegal principal (see below) so I assume that Talend is not able to read the krb5.ini / krb5.conf correctly although I've set the path to the file in Hadoop properties.
Could you please show me how to set up Hadoop properties right and maybe give me some other advice to get rid off the problem?
Thanks in advance!
Marc
Errors: (I renamed principal and realm due to data protection)
org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
    at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:53)
    at org.talend.designer.hdfsbrowse.hadoop.service.HadoopServiceBean.check(HadoopServiceBean.java:102)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckHadoopServicesDialog$5.run(CheckHadoopServicesDialog.java:373)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.talend.designer.hdfsbrowse.exceptions.HadoopServerException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
    at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:47)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider.checkService(AbstractCheckedServiceProvider.java:50)
    ... 5 more
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:206)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit.execute(CheckedWorkUnit.java:44)
    ... 6 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.talend.core.utils.ReflectionUtils.invokeStaticMethod(ReflectionUtils.java:229)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.provider.CheckedNamenodeProvider.check(CheckedNamenodeProvider.java:56)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.AbstractCheckedServiceProvider$1.run(AbstractCheckedServiceProvider.java:45)
    at org.talend.designer.hdfsbrowse.hadoop.service.check.CheckedWorkUnit$1.call(CheckedWorkUnit.java:65)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    ... 3 more
Caused by: java.io.IOException: Login failure for *PRINCIPAL@REALM* from keytab C:/Hadoop/Kerberos/*PRINCIPAL*.keytab: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name *PRINCIPAL@REALM*: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to *PRINCIPAL@REALM*
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:962)
    ... 12 more
Caused by: javax.security.auth.login.LoginException: java.lang.IllegalArgumentException: Illegal principal name *PRINCIPAL@REALM*: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to *PRINCIPAL@REALM*
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:588)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:953)
    ... 12 more
Caused by: java.lang.IllegalArgumentException: Illegal principal name *PRINCIPAL@REALM*: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to *PRINCIPAL@REALM*
    at org.apache.hadoop.security.User.<init>(User.java:50)
    at org.apache.hadoop.security.User.<init>(User.java:43)
    at org.apache.hadoop.security.UserGroupInformation$HadoopLoginModule.commit(UserGroupInformation.java:197)
    ... 24 more
Caused by: org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule: No rules applied to *PRINCIPAL@REALM*
    at org.apache.hadoop.security.authentication.util.KerberosName.getShortName(KerberosName.java:389)
    at org.apache.hadoop.security.User.<init>(User.java:48)
    ... 26 more
6 REPLIES
One Star

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

Try by giving the manual configurations. Don't use ambari to get the Configurations.
One Star

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

Even after manual configuration I'm still getting the same errors.
Any other ideas?
Employee

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

It looks like an issue with your k5b5 file, where did you place the krb5.ini/krb5.conf?
On windows by default it looks in C:\Windows and in linux under /etc/
One Star

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

I think you're right and the krb5.ini is the problem, but I already placed it in "C:\Windows".
In the krb5.ini I specified the path to the krb5cache and I checked that the cache-file is existing after I ordered the Kerberos ticket.
How can I set the path to look for the krb5.ini? Maybe the default has somehow been changed or the krb5.ini can't be found.
We tested the same configuration on another computer successfully - at least we can't spot any differences...
Since today I'm experiencing another problem: the Kerberos Ticket Manager doesn't show the tickets of the service user but that should not affect the previous problem because for that I used my user.
Looking forward to your answers!
One Star

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

The problem with the MIT Kerberos Ticket Manager doesn't exist anymore, about I've got still no workaround for the talend/kerberos-ticket problem.
One Star

Re: Establishing connection to Hadoop Cluster Hortonworks V2.2.3

I finally managed to create the connection to our cluster.
I only did a clean install of the correct JDK, deleted all other Java Versions and did a clean install of talend and re-checked the system variables.