One Star

Impala & HDFS error: java.lang.NoClassDefFoundError: org/htrace/Trace

Hi,
I'm using Talend real time big data edition.
I'm having an error when combining HDFSoutput components with Impala in the same job ( see scenario attached )
If we split reading from Impala and writing into HDFS into different jobs , it works fine
I do have a workaround but it would be great if we can use both in the same job.
Here is the error : 

Exception in thread "main" java.lang.NoClassDefFoundError: org/htrace/Trace
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:214)
at com.sun.proxy.$Proxy7.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:752)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy8.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2095)
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1214)
disconnected
disconnected
disconnected
disconnected
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1210)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1210)
at XX.bv_accountcoms_0_1.BV_AccountComs.tHDFSInput_3Process(BV_AccountComs.java:5696)



It looks like Impala loading a library or plugin that conflicts with HDFS libs
FYI : Talend version is 6.1.1
This is a standard ( not spark or mapreduce ) job 
2 REPLIES
Moderator

Re: Impala & HDFS error: java.lang.NoClassDefFoundError: org/htrace/Trace

Hi,
Did you post your whole job design screenshot into forum?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Impala & HDFS error: java.lang.NoClassDefFoundError: org/htrace/Trace

Yes that was it.