One Star

[resolved] Connect to Hortonworks sandbox under VirtualBox

I am running Hortonworks Sandbox 2 under VirtualBox. How do I connect the TOS Big Data 5.4.1 sandbox examples to this? Context variables set in the examples don't work. TOS is installed on windows 7 and sandbox is running on this.

  • Big Data
41 REPLIES
Moderator

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Hi janhess
We have reported your issue to our Big Data manager then come to you asap.
Thanks for posting your issue.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Hello,
Could you please give us more details? What does "Context variables don't work" mean? What error do you have?
Regards,
Rémy.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
: org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
I'm now trying hadoop 1.3.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

I can connect using 1.3 if I change all the context references for 'sandbox' to 127.0.0.1.
Running demo job Simple_Hive_row_input, it fails in tHiveRow_3 trying to load the data into the table with the error
: hive.ql.parse.ParseDriver - Parsing command: LOAD DATA LOCAL INPATH 'c:/temp/data_8XZF.txt' OVERWRITE INTO TABLE hivetest.users PARTITION (department='All')
: hive.ql.parse.ParseDriver - Parse Completed
FAILED: RuntimeException java.net.UnknownHostException: unknown host: sandbox.
The file c:/temp/data_8XZF.txt exists on my PC.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

The hortonworks sandbox is a CentOS VM right?
Regarding the second post, please use the hostnames and not the IPs.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

HortonWorks is running under VirtualBox centOS. If I use 'sandbox' as hostname nothing works.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Did you bind your hostname with the IP on your Windows hosts file ?
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Not sure. How do I do that?
The demo job creates an empty file on hadoop. It just fails to write any data to it. I'm wondering if it's to do with the query in tHiveRow_3.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

The hostname must be known by the client.
You will have to bind the hostname with the IP in the file: C:/Windows/System32/drivers/etc/hosts.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Well setting 127.0.0.1 to sandbox gets round the context file problem but the job still fails in tHiveRow_3.
connecting to socket on port 3685
connected
: org.apache.hadoop.hive.metastore.HiveMetaStore - 0: Opening raw store with implemenation classSmiley Surprisedrg.apache.hadoop.hive.metastore.ObjectStore
: org.apache.hadoop.hive.metastore.ObjectStore - ObjectStore, initialize called
: DataNucleus.Persistence - Property datanucleus.cache.level2 unknown - will be ignored
: org.apache.hadoop.hive.metastore.ObjectStore - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
: org.apache.hadoop.hive.metastore.ObjectStore - Initialized ObjectStore
: org.apache.hadoop.security.UserGroupInformation - No groups available for user njh85
: org.apache.hadoop.security.UserGroupInformation - No groups available for user njh85
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file \tmp\njh85\njh85_4752@TONTO_2014012309585770162539963460709.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: set hive.fetch.output.serde = org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file \tmp\njh85\njh85_4752@TONTO_2014012309585770162539963460709.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET mapred.job.map.memory.mb=1000
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file \tmp\njh85\njh85_4752@TONTO_2014012309585770162539963460709.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: SET mapred.job.reduce.memory.mb=1000
: org.apache.hadoop.hive.service.HiveServer - Putting temp output to file \tmp\njh85\njh85_4752@TONTO_2014012309585770162539963460709.pipeout
: org.apache.hadoop.hive.service.HiveServer - Running the query: drop database if exists hivetest cascade
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: hive.ql.parse.ParseDriver - Parsing command: drop database if exists hivetest cascade
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver - Starting command: drop database if exists hivetest cascade
: org.apache.hadoop.hive.ql.Driver -

: hive.metastore - Trying to connect to metastore with URI thrift://sandbox:9083
: org.apache.hadoop.security.UserGroupInformation - No groups available for user njh85
: hive.metastore - Waiting 1 seconds before next connection attempt.
: hive.metastore - Connected to metastore.
OK
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver - OK
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.service.HiveServer - Returning schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.service.HiveServer - Running the query: create database hivetest
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: hive.ql.parse.ParseDriver - Parsing command: create database hivetest
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver - Starting command: create database hivetest
: org.apache.hadoop.hive.ql.Driver -

OK
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver - OK
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.service.HiveServer - Returning schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.service.HiveServer - Running the query: create table hivetest.users (id int, first_name string, last_name string) partitioned by (department string) row format delimited fields terminated by '\;' stored as textfile
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: hive.ql.parse.ParseDriver - Parsing command: create table hivetest.users (id int, first_name string, last_name string) partitioned by (department string) row format delimited fields terminated by '\;' stored as textfile
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Starting Semantic Analysis
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Creating table hivetest.users position=13
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver - Starting command: create table hivetest.users (id int, first_name string, last_name string) partitioned by (department string) row format delimited fields terminated by '\;' stored as textfile
: org.apache.hadoop.hive.ql.Driver -

: hive.ql.exec.DDLTask - Default to LazySimpleSerDe for table hivetest.users
OK
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver - OK
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.service.HiveServer - Returning schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.service.HiveServer - Running the query: LOAD DATA LOCAL INPATH 'C:\tmp\data_CQJM.txt' OVERWRITE INTO TABLE hivetest.users PARTITION (department='All')
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: hive.ql.parse.ParseDriver - Parsing command: LOAD DATA LOCAL INPATH 'C:\tmp\data_CQJM.txt' OVERWRITE INTO TABLE hivetest.users PARTITION (department='All')
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:null, properties:null)
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver - Starting command: LOAD DATA LOCAL INPATH 'C:\tmp\data_CQJM.txt' OVERWRITE INTO TABLE hivetest.users PARTITION (department='All')
: org.apache.hadoop.hive.ql.Driver -

Copying data from file:/C:/tmp/data_CQJM.txt
: org.apache.hadoop.hive.ql.exec.Task - Copying data from file:/C:/tmp/data_CQJM.txt to hdfs://sandbox:8020/tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000
Copying file: file:/C:/tmp/data_CQJM.txt
: org.apache.hadoop.hive.ql.exec.Task - Copying file: file:/C:/tmp/data_CQJM.txt
: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 10.0.2.15:50010 java.net.ConnectException: Connection timed out: no further information
: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_-4349960216911638227_1318
: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 10.0.2.15:50010
: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
: org.apache.hadoop.hdfs.DFSClient - Error Recovery for blk_-4349960216911638227_1318 bad datanode nodes == null
: org.apache.hadoop.hdfs.DFSClient - Could not get block locations. Source file "/tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt" - Aborting...
Failed with exception java.io.IOException: File /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
: org.apache.hadoop.hive.ql.exec.Task - Failed with exception java.io.IOException: File /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask
Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTaskOK
: org.apache.hadoop.hive.ql.Driver - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.CopyTask
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.service.HiveServer - Running the query: select * from hivetest.users
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -
: hive.ql.parse.ParseDriver - Parsing command: select * from hivetest.users
: hive.ql.parse.ParseDriver - Parse Completed
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Starting Semantic Analysis
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Completed phase 1 of Semantic Analysis
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for source tables
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for subqueries
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Get metadata for destination tables
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Completed getting MetaData in Semantic Analysis
: org.apache.hadoop.hive.ql.ppd.OpProcFactory - Processing for FS(2)
: org.apache.hadoop.hive.ql.ppd.OpProcFactory - Processing for SEL(1)
: org.apache.hadoop.hive.ql.ppd.OpProcFactory - Processing for TS(0)
: org.apache.hadoop.hive.ql.parse.SemanticAnalyzer - Completed plan generation
: org.apache.hadoop.hive.ql.Driver - Semantic Analysis Completed
: org.apache.hadoop.hive.ql.exec.TableScanOperator - Initializing Self 0 TS
: org.apache.hadoop.hive.ql.exec.TableScanOperator - Operator 0 TS initialized
: org.apache.hadoop.hive.ql.exec.TableScanOperator - Initializing children of 0 TS
: org.apache.hadoop.hive.ql.exec.SelectOperator - Initializing child 1 SEL
: org.apache.hadoop.hive.ql.exec.SelectOperator - Initializing Self 1 SEL
: org.apache.hadoop.hive.ql.exec.SelectOperator - SELECT struct
: org.apache.hadoop.hive.ql.exec.SelectOperator - Operator 1 SEL initialized
: org.apache.hadoop.hive.ql.exec.SelectOperator - Initializing children of 1 SEL
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - Initializing child 3 OP
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - Initializing Self 3 OP
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - Operator 3 OP initialized
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - Initialization Done 3 OP
: org.apache.hadoop.hive.ql.exec.SelectOperator - Initialization Done 1 SEL
: org.apache.hadoop.hive.ql.exec.TableScanOperator - Initialization Done 0 TS
: org.apache.hadoop.hive.ql.Driver - Returning Hive schema: Schema(fieldSchemas:, properties:null)
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver - Starting command: select * from hivetest.users
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver - OK
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.ql.Driver -

: org.apache.hadoop.hive.service.HiveServer - Returning schema: Schema(fieldSchemas:, properties:null)
: org.apache.hadoop.hive.ql.exec.TableScanOperator - 0 finished. closing...
: org.apache.hadoop.hive.ql.exec.TableScanOperator - 0 forwarded 0 rows
: org.apache.hadoop.hive.ql.exec.SelectOperator - 1 finished. closing...
: org.apache.hadoop.hive.ql.exec.SelectOperator - 1 forwarded 0 rows
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - 3 finished. closing...
: org.apache.hadoop.hive.ql.exec.ListSinkOperator - 3 forwarded 0 rows
: org.apache.hadoop.hive.ql.exec.SelectOperator - 1 Close done
: org.apache.hadoop.hive.ql.exec.TableScanOperator - 0 Close done
: org.apache.hadoop.hive.ql.Driver -
: org.apache.hadoop.hive.ql.Driver -

disconnected
: org.apache.hadoop.hdfs.DFSClient - Failed to close file /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hive-njh85/hive_2014-01-23_09-58-38_544_8332326468037359341/-ext-10000/data_CQJM.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy8.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
Job Simple_hive_row_input ended at 09:59 23/01/2014.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

If I run the the job hdfs_input_output it creates an empty hadoop file then fails.
Starting job hdfs_input_output at 10:08 23/01/2014.
connecting to socket on port 3581
connected
: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 10.0.2.15:50010 java.net.ConnectException: Connection timed out: no further information
: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_2916331818421477522_1321
: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 10.0.2.15:50010
Exception in component tHDFSOutput_1
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hdp/test/data1.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hdp/test/data1.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
: org.apache.hadoop.hdfs.DFSClient - Error Recovery for blk_2916331818421477522_1321 bad datanode nodes == null
: org.apache.hadoop.hdfs.DFSClient - Could not get block locations. Source file "/user/hdp/test/data1.txt" - Aborting...
disconnected
: org.apache.hadoop.hdfs.DFSClient - Failed to close file /user/hdp/test/data1.txt
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hdp/test/data1.txt could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at com.sun.proxy.$Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3930)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3790)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)
Job hdfs_input_output ended at 10:09 23/01/2014.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Hi,
The problem comes from your chadoop cluster. It seems that the datanode (or maybe the tasktracker) is node available.
Cheers,
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

But I can create data files through a browser connected to the sandbox. It's just Talend that won't let me import or export data. I can create empty files.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Let's try to troubleshoot now.
": org.apache.hadoop.hdfs.DFSClient - Excluding datanode 10.0.2.15:50010"
What is the IP of the sandbox? It seems that the IP is 10.0.2.15 while you said 127.0.0.1. How is binded the sandbox hostname on the cluster side (in /etc/hosts) ? According to the IP, it seems you use a NAT connection between your local host and the VM. To be honest, using a Bridge or an internal network connection is much better.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

That doesn't work. Hortonworks sandbox fails to get an ip address and nothing connects to it.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

"Hortonworks sandbox fails to get an ip address and nothing connects to it."
So, of course Talend can't connect to it if there is no network between both machines.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

No it doesn't connect when set to Bridge or internal network. Only NAT seems to work.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

what's your IP in NAT ?
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

The ip for connection from a browser is 127.0.0.1:8888
I'm just using sandbox 1.3 'out of the box'.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

"From a browser"? Directly on the sandbox ? Or from your windows 7 ? The sandbox has a UI?
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

From windows 7.This is to run the Hortonworks tutorial. Note I changed the port from 8020 to 8888 in the previous post.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

How can you connect to the VM with the loopback address?
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

I don't understand. Sandbox says to connect the browser with 127.0.0.1:8888 which works fine.
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Can you say again the architecture?
You have a windows machine (windows seven) and you have downloaded the Sandbox VM which is a CentOS right? Your windows seven and the VM are connected together in NAT.
And you can connect to the VM using the address 127.0.0.1 ?
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

127.0.0.1:8888 from the browser. This is what sandbox vm tells me to use.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

Nobody got any ideas on how to make this work?
Employee

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

IMHO, it's definitely a problem of network in your VM.
One Star

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

So why will it create an empty file but not populate it?
Seventeen Stars

Re: [resolved] Connect to Hortonworks sandbox under VirtualBox

That could only work if there is a SSH tunnel but the sandbox does not establish such tunnel. Such tunnel can only be established be the user it self.
Where do you start the browser? In the sandbox or in Win7?