One Star

[resolved] Hive connector bug?

I keep getting errors similar to the following on a simple Hive job - this Null Pointer Exception looks like a bug (even if there is an incorrect setting it's not handling it properly). I've double checked the settings and they're the same as I use in my SQuirreL JDBC client which works... this is connecting to a Hortonworks cluster via HiveServer2:
Starting job Hive_Test at 15:31 18/08/2014.
connecting to socket on port 3508
connected
: org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:1158)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:246)
    at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:81)
    at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
: org.apache.hadoop.util.NativeCodeLoader - Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable
: hive.metastore - Trying to connect to metastore with URI thrift://hiveserver_host:10000
Exception in component tHiveConnection_1
java.lang.RuntimeException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.metastore.HiveMetaStoreClient
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:347)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
    at org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
    at org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
    at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
    at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
    at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
    at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
    at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at java.sql.DriverManager.getConnection(Unknown Source)
    at testproject.hive_test_0_1.Hive_Test.tHiveConnection_1Process(Hive_Test.java:371)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1063)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1413)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2444)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2456)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:341)
    ... 13 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1411)
    ... 18 more
Caused by: java.lang.NullPointerException
    at java.lang.ProcessBuilder.start(Unknown Source)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
    at org.apache.hadoop.util.Shell.run(Shell.java:418)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
    at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
    at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
    at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
    at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1418)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:352)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:214)
    ... 23 more
Exception in component tHiveInput_1
java.lang.NullPointerException
    at testproject.hive_test_0_1.Hive_Test.tHiveInput_1Process(Hive_Test.java:679)
    at testproject.hive_test_0_1.Hive_Test.runJobInTOS(Hive_Test.java:1075)
    at testproject.hive_test_0_1.Hive_Test.main(Hive_Test.java:922)
disconnected
Job Hive_Test ended at 15:31 18/08/2014.
3 REPLIES
One Star

Re: [resolved] Hive connector bug?

I got past this today, I switched from embedded to standalone mode, although I'm positive I tried that yesterday as well as a few other things. Anyway, I'll mark this as resolved, although it's also worth noting that I've had a situation where job runs didn't start and had blank output after clicking Run, with both Run and Kill dialogs greyed out, only closing the whole app and reopening it got the job working again (without any changes to the job itself).
One Star

Re: [resolved] Hive connector bug?

you need to add win utils and the same in job properties and done 
Moderator

Re: [resolved] Hive connector bug?

Hi,
Here is a KB article about:TalendHelpCenter:The missing winutils.exe program in the Big Data Jobs.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.