NoClassDefFoundError exception when running a Talend 6.3.1 Job with tHive components

Talend Version       6.3.1

Summary

 
Additional Versions  
Product Big Data
Component Hive
Problem Description

When running a Talend 6.3.1 Job that uses tHive components , it fails with the NoClassDefFoundError exception and the following Java Exception Stack:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException
at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:484)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:207)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:180)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at dwh_etl.generatedefaultmapvalues_9_1.generateDefaultMapValues.tELTHiveMap_1Process(generateDefaultMapValues.java:675)
at dwh_etl.generatedefaultmapvalues_9_1.generateDefaultMapValues.runJobInTOS(generateDefaultMapValues.java:2288)
at dwh_etl.generatedefaultmapvalues_9_1.generateDefaultMapValues.runJob(generateDefaultMapValues.java:2086)
at dwh_etl.transformhive_9_1.TransformHive.tRunJob_1Process(TransformHive.java:735)
at dwh_etl.transformhive_9_1.TransformHive.runJobInTOS(TransformHive.java:1531)
at dwh_etl.transformhive_9_1.TransformHive.main(TransformHive.java:1265)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more 

This issue occurs when launching this Job from Talend TAC 6.3.1, but it works when running from Studio. The same Job runs fine with Talend version 6.1.1.

Problem root cause

This issue is due to a bug. When building the Job, the jar file hive-metastore-1.2.1.2.3.2.0-2950.jar was incorrectly renamed -1.2.1.2.3.2.0-2950.jar and was stored in the lib directory of the Job build. As a consequence, the hive-metastore-1.2.1.2.3.2.0-2950.jar file that is used in the classpath of the Job cannot be found, and a java.lang.NoClassDefFoundError class is thrown when running the Job.

Solution or Workaround

This problem is fixed in Talend 6.4.1 and 6.3.2.

 

For Talend 6.3.1, the solution consists of applying Patch_20170607_TPS-1970_v1-6.3.1.zip. To apply this patch, perform the following steps:

  1. Contact Talend Support at https://www.talend.com/services/technical-support/ to request the patch.
  2. Once the patch has been assigned to you, download it by accessing https://talend-update.talend.com. Log in with your license credentials.
  3. Unzip the patch. Inside you will find tPigStoreResult_java.xml in plugins\org.talend.designer.components.bigdata_6.3.1.20161216_1026\components\tPigStoreResult.
  4. Make a backup of your existing tPigStoreResult_java.xml file.
  5. Copy the new tPigStoreResult_java.xml into these two locations:

    • the CommandLine installation directory: studio\plugins\org.talend.designer.components.bigdata_6.3.1.20161216_1026\components\tPigStoreResult
    • The Studio installation directory: plugins\org.talend.designer.components.bigdata_6.3.1.20161216_1026\components\tPigStoreResult

      In the new tPigStoreResult_java.xml file, the MVN=mvnSmiley Surprisedrg.talend.libraries/-1.2.1.2.3.2.0-2950/6.1.0 string that caused the issue should not be present.

  6. Restart Talend Studio and CommandLine.
Version history
Revision #:
11 of 11
Last update:
‎03-05-2018 01:06 PM
Updated by:
 
Contributors