'Can't get Kerberos realm' error when executing a Job that includes Kerberos-enabled Hadoop cluster objects on JobServer

Symptoms

A Job is created with a Hadoop cluster object, like HDFS, in Studio. The tHDFSConnection component is added to the Job. The Job runs successfully from Studio, however; the Job fails to execute from JobServer with the following error.

 

Exception in component tHDFSConnection_1
java.lang.IllegalArgumentException: Can't get Kerberos realm at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275) at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:260) at org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:337) at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:331) at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:941) at talendsvnprod.kerberoes_testing_0_1.kerberoes_testing.tHDFSConnection_1Process(kerberoes_testing.java:336) at talendsvnprod.kerberoes_testing_0_1.kerberoes_testing.runJobInTOS(kerberoes_testing.java:656) at talendsvnprod.kerberoes_testing_0_1.kerberoes_testing.main(kerberoes_testing.java:496) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84) at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63) ... 8 more
Caused by: KrbException: Cannot locate default realm at sun.security.krb5.Config.getDefaultRealm(Unknown Source)
... 14 more

 

Diagnosis

  1. The kinit command on Studio generates the Kerberos ticket, and the Job runs successfully. 
  2. The kinit command on JobServer failed with the error:
    Can't get Kerberos realm
  3. The issue is with the Kerberos configuration on JobServer; the krb5.conf file does not include the realm name.

Solution

To fix the issue, copy the krb5.conf file from the Kerberos server to the JobServer and run the Job.

 

Note:

  1. The krb5.conf file on the Kerberos server (KDC) is located in /etcThis file should be passed to Talend users for configuring Kerberos on JobServer and Studio machines. 
  2. If the JobServer is on a Windows machine, the krb5.conf file has to be renamed to krb.ini and should be placed in the C:\Windows directory.
Version history
Revision #:
6 of 6
Last update:
‎06-15-2018 12:30 PM
Updated by:
 
Contributors
Comments
rpatel

Can you please describe the solution a bit more? eg: From where to copy the file and where to place it? If the jobserver is running on Windows, then what should be the extension of that file and where to place the file. 

kalokala

@rpatel Changes are incorporated.