A Job with a Hadoop source or target fails when the Job is deployed from TAC

Symptoms

Any Job with a Hadoop source/target fails with an "Exception in thread "main" java.lang.NoClassDefFoundError: com/google/protobuf/ServiceException" error when deployed from TAC:

 

[INFO ]: demandcapture.hdfs_tac_connection_test_0_1.HDFS_TAC_Connection_TEST - TalendJob: 'HDFS_TAC_Connection_TEST' - Start.
[statistics] connecting to socket on port 10523
[statistics] connected
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[INFO ]: org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
[statistics] disconnected
[statistics] disconnected
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/protobuf/ServiceException
	at org.apache.hadoop.ipc.ProtobufRpcEngine.<clinit>(ProtobufRpcEngine.java:72)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Unknown Source)
	at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2138)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2103)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2197)
	at org.apache.hadoop.ipc.RPC.getProtocolEngine(RPC.java:205)
	at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:572)
	at org.apache.hadoop.hdfs.NameNodeProxies.createNNProxyWithClientProtocol(NameNodeProxies.java:418)
	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:314)
	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:708)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:651)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2696)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2733)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2715)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:382)
	at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:171)
	at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Unknown Source)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:168)
	at demandcapture.hdfs_tac_connection_test_0_1.HDFS_TAC_Connection_TEST.tHDFSList_1Process(HDFS_TAC_Connection_TEST.java:776)
	at demandcapture.hdfs_tac_connection_test_0_1.HDFS_TAC_Connection_TEST.tHDFSConnection_1Process(HDFS_TAC_Connection_TEST.java:474)
	at demandcapture.hdfs_tac_connection_test_0_1.HDFS_TAC_Connection_TEST.runJobInTOS(HDFS_TAC_Connection_TEST.java:2021)
	at demandcapture.hdfs_tac_connection_test_0_1.HDFS_TAC_Connection_TEST.main(HDFS_TAC_Connection_TEST.java:1838)
Caused by: java.lang.ClassNotFoundException: com.google.protobuf.ServiceException
	at java.net.URLClassLoader.findClass(Unknown Source)
	at java.lang.ClassLoader.loadClass(Unknown Source)
	at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
	at java.lang.ClassLoader.loadClass(Unknown Source)
	... 29 more

 

Diagnosis

The Job is successful from Studio but not from TAC. The issue is with the Nexus configuration on Studio: Nexus is not configured due to certain firewall issues.

This issue is observed when the jar file com.google.protobuf-2.4.0.jar is not available at the CommandLine library location C:\Talend\6.4.1\studio\configuration\.m2\repository\org\talend\libraries.

Solution

As Nexus is not available, the alternate method to fix this issue is to copy the missing jar (com.google.protobuf-2.4.0.jar) to Talend_Cmd_Installed_Directory\6.4.1\cmdline\studio\configuration\.m2\repository\org\talend\libraries and restart the CommandLine service/TAC.

Version history
Revision #:
2 of 2
Last update:
‎11-20-2017 05:08 PM
Updated by:
 
Contributors