Dependency issue with Talend Open Studio for Big Data 5.1

One Star

Dependency issue with Talend Open Studio for Big Data 5.1

I just download the recent version - TOS_BD-r84309-V5.1.1
I created a new project and imported the TALENDDEMOSJAVA project components. When I attempt to run any of the components like Oracle Output or MySQL Connection it gives a ClassNotFoundException looking for the driver classes. I see warnings in the IDE saying the some jars are missing.
Since these are just the built in components I assumed the dependencies would be included, but I am really new to TOS. Do I need to install something else first before the Big Data install?
Any ideas?
-Jonathan
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi Jonathan
Welcome to Talend Community!
I think what you need is TOS for DI which contains all jar files for DB connection.

Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

I went ahead and installed TOS for DI v5.1.1 yet when I attempt to run the following demo component I get the same ClassNotFoundException about the MySQL JDBC driver.
Job Designs > t01_Compo > Databases > Connection

Exception in component tMysqlConnection_1
java.lang.ClassNotFoundException: org.gjt.mm.mysql.Driver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.Class.forName0(Native Method)
connecting to socket on port 3758
connected
disconnected
at java.lang.Class.forName(Class.java:169)
at demo_project.connection_0_1.Connection.tMysqlConnection_1Process(Connection.java:348)
at demo_project.connection_0_1.Connection.runJobInTOS(Connection.java:1208)
at demo_project.connection_0_1.Connection.main(Connection.java:1076)
Job Connection ended at 22:08 05/08/2012.
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
Where do you install TOS?
Make sure there is no any spaces in the installation dir(e.g. C:\Program Files\TOS is forbidden), which may cuase ClassNotFoundException.
Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

I have both DI and BD installed in the following two directories with no spaces.
C:\bin\apps\talend\HDP-ETL-TOS_BD-V5.1.1
C:\bin\apps\talend\TOS_DI-Win32-r84309-V5.1.1
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

We tried adding the MySQL driver (mysql-connector-java-5.1.21-bin.jar) as a module but this didn't correct the issue.
We also tried manually copying the jars from DI as is mentioned in this forum post, (http://hortonworks.com/community/forums/topic/connecting-to-mysql-from-talend-open-studio-for-big-da...).
They are having the exact issue we are having but neither of the solutions mentioned worked for us.
Any other ideas?
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Do we need to have DI open first and then open BD? We have tried this and we still have the classpath issue. We have tried this with three people in three locations/environments so this can't be isolated to just us.
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
No. There is no need to install DI first.
According to your description, I don't know why it doesn't work. Simple we copy the jar file from TOS for DI and then save it into TOS for BD. Everything would be OK.
Could you tell us how do you install the jar file in TOS for BD?
Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

We have tried a couple different approaches. As I said earlier we tried adding the MySQL driver jars directly using the modules tab. It has a jar icon if you click that will allow you to select the jar file and then adds to the list/
We have also tried manually copying the jars from DI in the plugins directory as mentioned in the URL I listed earlier.
Is there another way to 'save' the jar(s) in BD?
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
All these two ways should work fine for TOS Big Data.
That's weird.
Are you sure that even though you are using TOS for DI directly this error also occurs?
Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Nope DI works fine.
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
Try the last work around.
Use tLibarryLoad to load this jar file at the beginning of this job.
Does it work fine? If so, it means this jar file is correct.
There must be some detail I miss in the process of installing jar file by using module view.
Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Using tLibraryLoad did the trick. Why doesn't BD package the jar files like DI?
Anyway thanks for your help.
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
I just download TOS for BD 5.1 and addthe MySQL driver jars directly using the modules tab succefully. That's so weird.
Regards,
Pedro
One Star

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Yeah we might have done something wrong while adding the jar through the modules tab. Who knows? I just realized the MySQL/Oracle jars are GPL and that is why you can't include with BD which is Apache 2.
Thanks again for your help. Very cool tool.
Employee

Re: Dependency issue with Talend Open Studio for Big Data 5.1

Hi
The Installation Guide has been updated for TOSBD 5.1 - http://talendforge.org/wiki/doku.php?id=doc:installation_guide#install_jar_dependencies, which explains the steps to install the 3rd party jar files.
Ciaran