Not able to connect to Hive on CDH 4.4

One Star

Not able to connect to Hive on CDH 4.4

Hi All,
I am unable to connect to HIve and perform actions.
Objective :
Create tables and load data into Hive.

Platform :
CDH 4.4 .
TOS for Big Data 5.4.0
Ubuntu OS.

Architecture :
Both CDH and TOS are on same machine(CDH with Single host implementation)
Components being used are :
tHiveConnections
tHiveCreateTable

Error Log Info:
Starting job Test at 08:06 13/11/2013.

connecting to socket on port 3895
connected
: org.apache.hadoop.hive.conf.HiveConf - hive-site.xml not found on CLASSPATH
: org.apache.hadoop.conf.Configuration - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2af081:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2af081:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation classSmiley Surprisedrg.apache.hadoop.hive.metastore.ObjectStore
: org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called
: DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@529c176c:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@529c176c:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
: org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
Hive history file=/tmp/root/hive_job_log_b7db2459-1147-4fdf-98c1-21bf81848390_1228259176.txt
: hive.ql.exec.HiveHistory - Hive history file=/tmp/root/hive_job_log_b7db2459-1147-4fdf-98c1-21bf81848390_1228259176.txt
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@52352d87:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@52352d87:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/b7db2459-1147-4fdf-98c1-21bf818483908013290778652005049.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: set hive.fetch.output.serde = org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/b7db2459-1147-4fdf-98c1-21bf818483908013290778652005049.pipeout
Hive history file=/tmp/root/hive_job_log_5dcb58b9-ff3b-41bb-b405-4cc6c5968577_774338328.txt
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@31923ca5:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@31923ca5:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: hive.ql.exec.HiveHistory - Hive history file=/tmp/root/hive_job_log_5dcb58b9-ff3b-41bb-b405-4cc6c5968577_774338328.txt
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@251c135c:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@251c135c:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/5dcb58b9-ff3b-41bb-b405-4cc6c596857743230144672065683.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: set hive.fetch.output.serde = org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/5dcb58b9-ff3b-41bb-b405-4cc6c596857743230144672065683.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: use ticket_details
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=Driver.run>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=TimeToSubmit>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=compile>
: hive.ql.parse.ParseDriver - Parsing command: use ticket_details
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=compile start=1384330016525 end=1384330016804 duration=279>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=Driver.execute>
: org.apache.hadoop.conf.Configuration - mapred.job.name is deprecated. Instead, use mapreduce.job.name
: org.apache.hadoop.hive.ql.Driver - Starting command: use ticket_details
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=TimeToSubmit start=1384330016525 end=1384330016821 duration=296>
: hive.metastore - Trying to connect to metastore with URI thrift://localhost:7432
: hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:2563)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:2549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:293)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:163)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1100)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2219)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2230)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1116)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1105)
at org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3533)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:239)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:516)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:410)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:782)
at demobigdata.test_0_1.Test.main(Test.java:647)
: hive.metastore - Waiting 1 seconds before next connection attempt.
: hive.metastore - Connected to metastore.
: org.apache.hadoop.hive.metastore.RetryingMetaStoreClient - MetaStoreClient lost connection. Attempting to reconnect.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:412)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:399)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:736)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at $Proxy10.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1116)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1105)
at org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3533)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:239)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:516)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:410)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:782)
at demobigdata.test_0_1.Test.main(Test.java:647)
: hive.metastore - Trying to connect to metastore with URI thrift://localhost:7432
: hive.metastore - set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.
org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:2563)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:2549)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:293)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.reconnect(HiveMetaStoreClient.java:191)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:87)
at $Proxy10.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1116)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1105)
at org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3533)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:239)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:516)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:410)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:782)
at demobigdata.test_0_1.Test.main(Test.java:647)
: hive.metastore - Waiting 1 seconds before next connection attempt.
FAILED: Error in metadata: org.apache.thrift.transport.TTransportException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
: hive.metastore - Connected to metastore.
: org.apache.hadoop.hive.ql.exec.Task - FAILED: Error in metadata: org.apache.thrift.transport.TTransportException
org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.thrift.transport.TTransportException
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1120)
at org.apache.hadoop.hive.ql.metadata.Hive.databaseExists(Hive.java:1105)
at org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3533)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:239)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:516)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:410)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:782)
at demobigdata.test_0_1.Test.main(Test.java:647)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:412)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:399)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:736)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at $Proxy10.getDatabase(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getDatabase(Hive.java:1116)
... 16 more
: org.apache.hadoop.hive.ql.Driver - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=Driver.execute start=1384330016804 end=1384330019913 duration=3109>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=releaseLocks>
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=releaseLocks start=1384330019913 end=1384330019913 duration=0>
Exception in component tHiveCreateTable_2
java.sql.SQLException: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:194)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:516)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:410)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:782)
at demobigdata.test_0_1.Test.main(Test.java:647)
disconnected
Job Test ended at 08:07 13/11/2013.
Employee

Re: Not able to connect to Hive on CDH 4.4

Hello,
I'm surprised your metastore to be running on the port 7432. Can you please confirm that? Can you please edit your hive-site.xml on the cluster side in order to get the information?
Cheers,
Rémy.
One Star

Re: Not able to connect to Hive on CDH 4.4

Hi Remy ,
Thanks for looking into it . I am not sure on what port is it running. I am using Cloudera Hadoop 4.4. According to document its 9083 . Thanks for bringing this up . Let me try and get back
One Star

Re: Not able to connect to Hive on CDH 4.4

Hi Remy ,
With the changes made .. I am getting below error .
Starting job Test at 10:50 13/11/2013.

connecting to socket on port 4002
connected
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.hadoop.hive.conf.HiveConf - hive-site.xml not found on CLASSPATH
: org.apache.hadoop.conf.Configuration - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@7e717d91:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@7e717d91:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation classSmiley Surprisedrg.apache.hadoop.hive.metastore.ObjectStore
: org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called
: DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@32fd4662:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@32fd4662:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
: org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
Hive history file=/tmp/root/hive_job_log_c59af570-aca6-4a1a-8db6-d8181b24cb23_379283623.txt
: hive.ql.exec.HiveHistory - Hive history file=/tmp/root/hive_job_log_c59af570-aca6-4a1a-8db6-d8181b24cb23_379283623.txt
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2d4c8822:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2d4c8822:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/c59af570-aca6-4a1a-8db6-d8181b24cb23291922524543407485.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: set hive.fetch.output.serde = org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/c59af570-aca6-4a1a-8db6-d8181b24cb23291922524543407485.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET mapreduce.framework.name=yarn
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/c59af570-aca6-4a1a-8db6-d8181b24cb23291922524543407485.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET yarn.resourcemanager.address=mlbis.local:8032
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/c59af570-aca6-4a1a-8db6-d8181b24cb23291922524543407485.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: use ticket_details
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=Driver.run>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=TimeToSubmit>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=compile>
: hive.ql.parse.ParseDriver - Parsing command: use ticket_details
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=compile start=1384339814199 end=1384339814491 duration=292>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=Driver.execute>
: org.apache.hadoop.conf.Configuration - mapred.job.name is deprecated. Instead, use mapreduce.job.name
: org.apache.hadoop.hive.ql.Driver - Starting command: use ticket_details
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=TimeToSubmit start=1384339814199 end=1384339814508 duration=309>
: hive.metastore - Trying to connect to metastore with URI thrift://localhost:9083
: hive.metastore - Waiting 1 seconds before next connection attempt.
: hive.metastore - Connected to metastore.
FAILED: Error in metadata: ERROR: The database ticket_details does not exist.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Exception in component tHiveLoad_1
java.sql.SQLException: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:194)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHDFSInput_1Process(Test.java:445)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:734)
at demobigdata.test_0_1.Test.main(Test.java:599)
: org.apache.hadoop.hive.ql.exec.Task - FAILED: Error in metadata: ERROR: The database ticket_details does not exist.
org.apache.hadoop.hive.ql.metadata.HiveException: ERROR: The database ticket_details does not exist.
at org.apache.hadoop.hive.ql.exec.DDLTask.switchDatabase(DDLTask.java:3534)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:239)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:982)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHDFSInput_1Process(Test.java:445)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:734)
at demobigdata.test_0_1.Test.main(Test.java:599)
: org.apache.hadoop.hive.ql.Driver - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=Driver.execute start=1384339814491 end=1384339815601 duration=1110>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=releaseLocks>
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=releaseLocks start=1384339815601 end=1384339815601 duration=0>
disconnected
Job Test ended at 10:50 13/11/2013.
One Star

Re: Not able to connect to Hive on CDH 4.4

HI ,
I feel TOS is running wring query , error log clearly states that the query used for creating table is not working . I tried same query on hive manually and got same set of error .
Wrong Query :
CREATE TABLE tkt_details() PARTITIONED BY (DeviceName null,TicketCount null) ROW FORMAT DELIMITED FIELDS TERMINATED BY ';' LINES TERMINATED BY '
' STORED AS TEXTFILE


Error Log:

Starting job Test at 05:58 14/11/2013.

connecting to socket on port 3360
connected
: org.apache.hadoop.hive.conf.HiveConf - hive-site.xml not found on CLASSPATH
: org.apache.hadoop.conf.Configuration - mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
: org.apache.hadoop.conf.Configuration - mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
: org.apache.hadoop.conf.Configuration - mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2b03be0:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@2b03be0:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation classSmiley Surprisedrg.apache.hadoop.hive.metastore.ObjectStore
: org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called
: DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@bf47ae8:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@bf47ae8:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
: org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
Hive history file=/tmp/root/hive_job_log_48656671-33ae-47b5-9596-f05bd87a9841_886133173.txt
: hive.ql.exec.HiveHistory - Hive history file=/tmp/root/hive_job_log_48656671-33ae-47b5-9596-f05bd87a9841_886133173.txt
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@7371b4ca:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
: org.apache.hadoop.conf.Configuration - org.apache.hadoop.hive.conf.LoopingByteArrayInputStream@7371b4ca:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
: org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/48656671-33ae-47b5-9596-f05bd87a98412321908055283865130.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: set hive.fetch.output.serde = org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/48656671-33ae-47b5-9596-f05bd87a98412321908055283865130.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET mapreduce.framework.name=yarn
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/48656671-33ae-47b5-9596-f05bd87a98412321908055283865130.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET yarn.resourcemanager.address=mlbis.local:8032
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file /tmp/root/48656671-33ae-47b5-9596-f05bd87a98412321908055283865130.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: CREATE TABLE tkt_details() PARTITIONED BY (DeviceName null,TicketCount null) ROW FORMAT DELIMITED FIELDS TERMINATED BY ';' LINES TERMINATED BY '
' STORED AS TEXTFILE
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=Driver.run>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=TimeToSubmit>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=compile>
: hive.ql.parse.ParseDriver - Parsing command: CREATE TABLE tkt_details() PARTITIONED BY (DeviceName null,TicketCount null) ROW FORMAT DELIMITED FIELDS TERMINATED BY ';' LINES TERMINATED BY '
' STORED AS TEXTFILE
FAILED: ParseException line 1:27 mismatched input ')' expecting Identifier near '(' in column specification
Query returned non-zero code: 40000, cause: FAILED: ParseException line 1:27 mismatched input ')' expecting Identifier near '(' in column specification
: org.apache.hadoop.hive.ql.Driver - FAILED: ParseException line 1:27 mismatched input ')' expecting Identifier near '(' in column specification
org.apache.hadoop.hive.ql.parse.ParseException: line 1:27 mismatched input ')' expecting Identifier near '(' in column specification
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:446)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:441)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
at demobigdata.test_0_1.Test.tHiveCreateTable_2Process(Test.java:536)
at demobigdata.test_0_1.Test.tHiveConnection_1Process(Test.java:429)
at demobigdata.test_0_1.Test.runJobInTOS(Test.java:785)
at demobigdata.test_0_1.Test.main(Test.java:650)
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=compile start=1384408687697 end=1384408687925 duration=228>
: org.apache.hadoop.hive.ql.Driver - <PERFLOG method=releaseLocks>
: org.apache.hadoop.hive.ql.Driver - </PERFLOG method=releaseLocks start=1384408687925 end=1384408687925 duration=0>
disconnected
Job Test ended at 05:58 14/11/2013.
One Star

Re: Not able to connect to Hive on CDH 4.4

Sorry , it's my bad.. i had not mentioned the fields which caused above error