Sqoop import failing as HIVE_CONF_DIR not set

Four Stars

Sqoop import failing as HIVE_CONF_DIR not set

Hello everyone,

 

I have installed Talend Data Fabric 6.4 on my Windows 7 desktop  and created a Sqoop job to import a table from MS SQL server into Cloudera Hadoop Data lake cluster (CDH5.14).

 

The job is simple and quick and I have created a new Metadata -> Hadoop cluster for Sqoop job to use. Then used the in-built Hadoop cluster in Sqoop job as below.

 

Talend_Sqoop_Job_Config_1.JPG

 

 

 

I also added Advanced Settings -> Additional Arguments to import the table into Hive metastore and create a Hive table automatically as shown below.

Talend_Sqoop_Additional_Config.JPG

 

 

When I run the job the Sqoop import completed successfully however when loading the content and creating Hive table the job failed with below exception

 

 

[ERROR]: org.apache.sqoop.hive.HiveConfig - Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.

[ERROR]: org.apache.sqoop.tool.ImportTool - Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf

 

Spoiler
 

I even downloaded the Apache hive and created HIVE_CONF_DIR environment variable in Windows advanced properties and ran the job but still the job failed.

 

Also FYI, I have manually ran a Sqoop job from Edge node on Data lake cluster using Sqoop CLI (sqoop import) and it executed successfully including HIVE import so I believe the cluster is configured correctly.

 

Can someone please advice what's wrong with Sqoop job import and why its complaining HIVE_CONF error??

 

Thanks,

Mahesh

 

Moderator

Re: Sqoop import failing as HIVE_CONF_DIR not set

Hello,

Here is a related article about:https://community.cloudera.com/t5/Batch-SQL-Apache-Hive/Error-with-quot-Make-sure-HIVE-CONF-DIR-is-s...

Let us know if the solution works for you.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Sqoop import failing as HIVE_CONF_DIR not set

Hi Sabrina,

 

Thanks for reply but it didn't resolve the issue. I have modified Hadoop Classpath variable as per the given link and tried running Sqoop but the error not resolved.

Talend_Hadoop_Claspath.JPG

 

The provided link is talking about the Sqoop import issue with command line running from a UNIX server however this issue is slightly different as I am running Sqoop from Talend in from Windows PC.

 

Thanks,

Mahesh

 

 

Moderator

Re: Sqoop import failing as HIVE_CONF_DIR not set

Hello,

With your Talend Data Fabric 6.4, could you please create a case on talend support portal so that we can give you a remote assistance (webex seesion) through support cycle with priority?

https://login.talend.com/support-login.php.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.