One Star

Problem to establish a connection with tHiveConnection

Hi,
I have a VM HortonWorks (HDP 2.2) and Talend (TOS 5.6.1).
I try to make a connection with a tHiveConnection component. It doesn't work !!
I don't know why, can you help me ? 
Thanks
Marc
See below my log and a screenshot of my parameter

Démarrage du job a a 18:48 14/03/2015.
connecting to socket on port 3432
connected
: org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:1158)
at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:246)
at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:81)
at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at bd_mpar.a_0_1.a.tHiveConnection_1Process(a.java:336)
at bd_mpar.a_0_1.a.runJobInTOS(a.java:643)
at bd_mpar.a_0_1.a.main(a.java:500)
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: hive.metastore - Trying to connect to metastore with URI thrift://sandbox:10000
Exception in component tHiveConnection_1
java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:347)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:292)
at org.apache.hive.service.cli.session.SessionManager.applyAuthorizationConfigPolicy(SessionManager.java:88)
at org.apache.hive.service.cli.session.SessionManager.init(SessionManager.java:63)
at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
at org.apache.hive.service.cli.CLIService.init(CLIService.java:110)
at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.<init>(EmbeddedThriftBinaryCLIService.java:35)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:164)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
disconnected
at bd_mpar.a_0_1.a.tHiveConnection_1Process(a.java:336)
at bd_mpar.a_0_1.a.runJobInTOS(a.java:643)
at bd_mpar.a_0_1.a.main(a.java:500)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1413)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2444)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2456)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:341)
... 13 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1411)
... 18 more
Caused by: java.lang.NullPointerException
at java.lang.ProcessBuilder.start(Unknown Source)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:139)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1418)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:352)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:214)
... 23 more
Job a terminé à 18:48 14/03/2015.

4 REPLIES
One Star

Re: Problem to establish a connection with tHiveConnection

a screenshot of my parameter

Employee

Re: Problem to establish a connection with tHiveConnection

If you are attempting to connect your studio from your local machine to the Sandbox you need to make sure the networks and hostnames on both your local machine the host has an entry for the VM and the IP for the Sandbox with the correct DNS name.  There are few other areas that you may need to update for your machine, it really depends on your PC or Mac that you are using and how to manage it.
Employee

Re: Problem to establish a connection with tHiveConnection

Hi.  Please update your tHiveConnection properties:  
1.  In the Version section:  Change Distribution to "Custom - Unsupported" and Check the "Use Yarn" Checkbox
2. In the Connection Section:  Change Connection Mode to "Standalone"
If you are creating the connection within the Sandbox, uncheck the "Set Resource Manager" and "Set Namenode URI" in the Hadoop Properties section.
See screenshot:

Re: Problem to establish a connection with tHiveConnection

Hi All,
I am using Talend Open Studio (TOS_DI-20141207_1530-V5.6.1)  and VM Cloudera (Hadoop 0.20.2-cdh3u2),trying to make connection using tHiveConnection ,but it is not working as shown below .

And getting below shown error in tHiveConnection 
could you please help me