Getting java.lang.NoClassDefFoundError for custom component on jobserver

Highlighted
Five Stars

Getting java.lang.NoClassDefFoundError for custom component on jobserver

We successfully created a custom component to read from rabbitMQ using the new framework

The component works in the studio 7.0.1.

But when we run the job on the jobserver we get an error - see log below;

 

Are there any additional steps when we publish/deploy custom components?

 

### Job STARTED at 2018/12/09 15:28:08 (jobId=20181209_152803_5VNKG, jobExecutionId=20181209152808_pGOkV) ###
[statistics] disconnected
[statistics] disconnected
Exception in thread "main" java.lang.NoClassDefFoundError: com/rabbitmq/client/DefaultConsumer
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetPublicMethods(Class.java:2902)
at java.lang.Class.getMethods(Class.java:1615)
at org.talend.sdk.component.runtime.manager.asm.ProxyGenerator.hasInterceptors(ProxyGenerator.java:491)
at org.talend.sdk.component.runtime.manager.ComponentManager.handleProxy(ComponentManager.java:921)
at org.talend.sdk.component.runtime.manager.ComponentManager.access$1400(ComponentManager.java:159)
at org.talend.sdk.component.runtime.manager.ComponentManager$Updater.lambda$onCreate$14(ComponentManager.java:1076)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.talend.sdk.component.runtime.manager.ComponentManager$Updater.onCreate(ComponentManager.java:1068)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.lambda$null$1(ContainerManager.java:370)
at org.talend.sdk.component.container.ContainerManager.safeInvoke(ContainerManager.java:144)
at org.talend.sdk.component.container.ContainerManager.access$1100(ContainerManager.java:54)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.lambda$create$2(ContainerManager.java:370)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.talend.sdk.component.container.ContainerManager$ContainerBuilder.create(ContainerManager.java:371)
at org.talend.sdk.component.runtime.manager.ComponentManager.addPlugin(ComponentManager.java:711)
at org.talend.sdk.component.runtime.manager.ComponentManager$3.lambda$new$3(ComponentManager.java:377)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.talend.sdk.component.runtime.manager.ComponentManager$3.<init>(ComponentManager.java:377)
at org.talend.sdk.component.runtime.manager.ComponentManager.instance(ComponentManager.java:349)
### Job ENDED WITH ERROR at 2018/12/09 15:28:09 (jobId=20181209_152803_5VNKG, jobExecutionId=20181209152808_pGOkV) ###

at talend.prollyaanvraag_0_1.pRollyAanvraag.AMQPInput_1Process(pRollyAanvraag.java:1148)
at talend.prollyaanvraag_0_1.pRollyAanvraag.tLibraryLoad_1Process(pRollyAanvraag.java:672)
at talend.prollyaanvraag_0_1.pRollyAanvraag.runJobInTOS(pRollyAanvraag.java:2240)
at talend.prollyaanvraag_0_1.pRollyAanvraag.main(pRollyAanvraag.java:1977)
Caused by: java.lang.ClassNotFoundException: com.rabbitmq.client.DefaultConsumer
at org.talend.sdk.component.classloader.ConfigurableClassLoader.loadClass(ConfigurableClassLoader.java:218)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 52 more

Highlighted
Employee

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

Hello,
It seems that you have a missing dependency in your project. the one that should provide the class `com/rabbitmq/client/DefaultConsumer`.
Can you check your dependencies and ensure you have all your required dependencies in compile scope.
if you plan to use any provided dependencies ensure they are provided by the framework or by the running platform (job server).

An easy way to check your job dependencies is to build your job from the studio by  right click on the job => Build Job and check the lib folder to ensure your jars files are all in there.

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

Anas,

 

Thanks for getting back.

I can run the job on the client machine and then it works.

But when I deploy the job from the TAC to the jobserver, it fails.

This is the .bat file on the jobserver

%~d0
cd %~dp0
D:\Java\jdk1.8.0_191\bin\java.exe -Xms256M -Xmx1024M -cp .;../../../cache/lib/3569584080_551552229/routines.jar;../../../cache/lib/2515122170_1359808236/amqp-client-5.4.3.jar;../../../cache/lib/2053048282_4182183202/advancedPersistentLookupLib-1.2.jar;../../../cache/lib/3434323647_3256683220/amqp-component-0.0.3-SNAPSHOT.jar;../../../cache/lib/1336530737_2793933396/commons-collections-3.2.2.jar;../../../cache/lib/242457337_1697318851/component-api-0.0.4.jar;../../../cache/lib/1086752861_2225934399/component-runtime-design-extension-0.0.5.jar;../../../cache/lib/746244976_2978155144/component-runtime-di-0.0.5.jar;../../../cache/lib/273176393_3420392301/component-runtime-impl-0.0.5.jar;../../../cache/lib/3859498358_4097051798/component-runtime-manager-0.0.5.jar;../../../cache/lib/2549687820_1712981974/component-spi-0.0.5.jar;../../../cache/lib/2754408678_1847134243/container-core-0.0.5.jar;../../../cache/lib/2178828168_605815915/dom4j-1.6.1.jar;../../../cache/lib/1972797759_4275703019/geronimo-annotation_1.3_spec-1.0.jar;../../../cache/lib/1732982708_2886797928/geronimo-json_1.1_spec-1.0.jar;../../../cache/lib/1039009832_3090090697/geronimo-jsonb_1.0_spec-1.0.jar;../../../cache/lib/2144549254_647365790/jaxen-1.1.1.jar;../../../cache/lib/2580262047_1866991928/jboss-serialization.jar;../../../cache/lib/2295871601_1549535082/johnzon-core-1.1.7.jar;../../../cache/lib/941375462_1716848605/johnzon-jsonb-1.1.7.jar;../../../cache/lib/1833419795_1878591558/johnzon-mapper-1.1.7.jar;../../../cache/lib/2437169554_1109001852/jtds-1.3.1-patch.jar;../../../cache/lib/2065431124_674301086/log4j-1.2.15.jar;../../../cache/lib/1015062404_674628767/log4j-1.2.16.jar;../../../cache/lib/3107690740_1118897678/slf4j-api-1.7.25.jar;../../../cache/lib/2601911240_1579091735/slf4j-log4j12-1.7.25.jar;../../../cache/lib/221808531_3912436596/talend_DB_mssqlUtil-1.2-20171017.jar;../../../cache/lib/237199807_3543468909/talend_file_enhanced_20070724.jar;../../../cache/lib/1009653832_312279964/trove.jar;../../../cache/lib/849693260_1937901690/xbean-asm6-shaded-4.7.jar;../../../cache/lib/1858677389_2343373179/xbean-finder-shaded-4.7.jar;../../../cache/lib/1125810284_1424230226/xbean-reflect-4.7.jar;../../../cache/lib/1420748432_1722746990/prollyaanvraag_0_1.jar;../../../cache/lib/1908744071_2077034843/prollyinformatie_0_1.jar talend.prollyaanvraag_0_1.pRollyAanvraag --context=PY910 %* --resuming_logs_dir_path=D:\Talend\TalendJobServersFiles\jobexecutions/logs\20181214155850_YKTnb\resuming_20181214155850_YKTnb.log --pid=20181214155850_YKTnb --client_host=10.30.2.117 --log4jLevel=Info

 

It has all the jars but somehow it does not recognise them.

Can it be related to the new component framework, because we used it to build a custom component?

Highlighted
Employee

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

Hello,
Thank you for sharing your class path.

I can see from your class path that you are using an old version (0.0.4) of the framework. This version is having a known issue about loading dependencies from the job lib folder that was fixed in 1.0.4
You can find the framework changelog in the doc : https://talend.github.io/component-runtime the issue is referenced in Talend jira under the id TCOMP-922.

I strongly suggest you to upgrade the framework version to get the fix.


Direct link to the changelog for the latest released version https://talend.github.io/component-runtime/main/1.1.4/changelog.html

You can also check the compatibility matrix (studio/framework) https://talend.github.io/component-runtime/main/1.1.4/compatibility.html

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

HI,

I have the same Issue:

 

Everything is ok with my new component on the studio. 

It's also working when I launch the job manuelly from the exported nexuse release, using the .bat script.

 

But when I deploy the nexus release on Tac/jobserver, I have the execption java.lang.NoClassDefFoundError: com/google/api/client/http/AbstractInputStreamContent.

 

I'm using the version 1.1.1 of the framework wich is compatible with my studio Talend MDM platform 7.1.1.

 

Any solution/workaround?

 

Thanks.

 

Youssef

 

 

Highlighted
Five Stars

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

I've tried to use diffrent versions of the framework but Tac says that it can't deploy jobs comming from a non supported version of studio...

 

I endup using a tjava with the component logic as an external library.

 

It's very annoying to waste time understanding and developping over a framework that can't go on production. Smiley Mad

 

 

Highlighted
Moderator

Re: Getting java.lang.NoClassDefFoundError for custom component on jobserver

Hello,

 

Have you already checked the configuration on the TAC navigating to command line and make sure it is pointing to the folder the custom component is on? Please ensure that you copy the entire custom component folder to that directory.

1.png_20150624-1249.png

 

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Downloads and Trials

Test drive Talend's enterprise products.

Downloads