Error while running sqoop using talend

One Star

Error while running sqoop using talend

I am using Talend 5.3.1 open studio for big data.I am trying to import data from SQL server to HDFS using sqoop
but when i am running the job i get the following error
"ERROR security.UserGroupInformation: PriviledgedActionException "
How to solve this error?
I have face this same problem when i was running MR jobs using eclipse so when i used the below code the error was resolved.
"UserGroupInformation ugi = UserGroupInformation.createRemoteUser("hduser");"
So is there any option to create remote user.
Employee

Re: Error while running sqoop using talend

Hello,
Don't you have a more detailed exception?
What is the confirguration of your Sqoop component? Which Sqoop component do you use?
Cheers,
One Star

Re: Error while running sqoop using talend

I am using CDH4.5 Sqoop component the the detailed error is
"14/03/03 17:22:02 ERROR security.UserGroupInformation: PriviledgedActionException as:296922 (authSmiley FrustratedIMPLE) cause:java.net.ConnectException: Call From INFPH01332/localhost to localhost:8021 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
14/03/03 17:22:02 ERROR tool.ImportTool: Encountered IOException running import job: java.net.ConnectException: Call From INFPH01332/localhost to localhost:8021 failed on connection exception: java.net.ConnectException: Connection refused: no further information; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1165)
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
at org.apache.hadoop.mapred.$Proxy11.getStagingAreaDir(Unknown Source)
at org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1223)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:481)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:511)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:163)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:141)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:211)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465)
at org.apache.sqoop.manager.SQLServerManager.importTable(SQLServerManager.java:99)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_2Process(SqoopConnect.java:641)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_2Process(SqoopConnect.java:563)
at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)
Caused by: java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:524)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:472)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:566)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:215)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1271)
at org.apache.hadoop.ipc.Client.call(Client.java:1141)
... 24 more
Exception in component tSqoopImport_2
java.lang.Exception: The Sqoop import job has failed
at sampleproj.sqoopconnect_0_1.SqoopConnect.tSqoopImport_2Process(SqoopConnect.java:643)
at sampleproj.sqoopconnect_0_1.SqoopConnect.tLibraryLoad_2Process(SqoopConnect.java:563)
at sampleproj.sqoopconnect_0_1.SqoopConnect.runJobInTOS(SqoopConnect.java:877)
at sampleproj.sqoopconnect_0_1.SqoopConnect.main(SqoopConnect.java:743)"
Employee

Re: Error while running sqoop using talend

Hi,
Your Talend studio is installed on the same machine than the cluster?
One Star

Re: Error while running sqoop using talend

Hi,
I have recently resolved this issue with my sqoopimport in talend.
This issue is coming when your jobtracker is not able to communicate to namenode.
is your hadoop cluster a single node cluster or multiple node?
can you locate your core-site.xml file?. as i have installed cloudera manager my core-site.xml was existing in
/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop-0.20-mapreduce/example-confs/conf.pseudo/core-site.xml
edit the above file. by default the fs.defaultname is always hdfs://localhost:8020.
change it to the ipaddress of your namenode.
<property>
<name>fs.default.name</name>
<value>hdfs://10.176.381.211:8020</value>
</property>
restart your hdfs from cloudera manager.
This should resolve the connection refused error and at least move you forward.
Regards
Gokul
One Star

Re: Error while running sqoop using talend

My Talend studio is installed on other machine than the cluster.
I have also edited the core-site.xml , replacing the localhost to ip- address
One Star

Re: Error while running sqoop using talend

Hi,
after the changes are you still getting the same error?
how you are executing your job?
can you use the oozie tab in talend to run the job on the hadoop cluster?
Thanks
One Star

Re: Error while running sqoop using talend

Hi Team, 
I am facing the below error when i am trying to sqoop data from MySQL DB to HDFS (Cloudera quickstart for cdh5.8) which is a single node cluster. Version is TOS_BD-V6.3.0. Had tried using both JAVA API and Commandline options but no luck. Connections with HDFS, Hive and MySQL DB are successful.
My job is as follows : tLibraryLoad --> tSqoopImport
When i use JAVA API : : org.apache.sqoop.tool.ImportTool - Encountered IOException running import job: java.io.IOException: Job status not available
When i use Command line :: java.io.IOException: Cannot run program "sqoop": CreateProcess error=2, The system cannot find the file specified
One Star

Re: Error while running sqoop using talend

Hi,
Your Talend studio is installed on the same machine than the cluste

I installed Hadoop 1.1.2 and Hive 0.9.0. I didn't set username and password when i installed Hive.What can i set in the  username and password part?