[resolved] HiveDB Connection needs my own SerDe

One Star

[resolved] HiveDB Connection needs my own SerDe

Hi. I'm trying to connect Open studio to Hive.

In Hive i have table with custom defined fields.
Open studio find table without problems, but if I try to Retrieve Schema i get such error:


pasteboard.co/NKgzjal.png

And in talend log i get:

!ENTRY org.talend.platform.logging 4 0 2015-09-21 10:49:12.375
!MESSAGE 2015-09-21 10:49:12,372 ERROR org.talend.commons.exception.CommonExceptionHandler  - java.lang.reflect.InvocationTargetException

!STACK 0
java.sql.SQLException: java.lang.reflect.InvocationTargetException
at org.talend.metadata.managment.hive.EmbeddedHiveDataBaseMetadata.getColumns(EmbeddedHiveDataBaseMetadata.java:401)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.getColumnsResultSet(ExtractManager.java:844)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.extractColumns(ExtractManager.java:641)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.returnMetadataColumnsFormTable(ExtractManager.java:521)
at org.talend.core.model.metadata.builder.database.ExtractMetaDataFromDataBase.returnMetadataColumnsFormTable(ExtractMetaDataFromDataBase.java:224)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm.pressRetreiveSchemaButton(DatabaseTableForm.java:1150)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm.access$13(DatabaseTableForm.java:1121)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm$4.widgetSelected(DatabaseTableForm.java:795)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4454)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1388)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3799)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3409)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:832)
at org.eclipse.jface.window.Window.open(Window.java:808)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction.handleWizard(AbstractCreateTableAction.java:139)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction$1$1.run(AbstractCreateTableAction.java:1088)
at org.talend.repository.RepositoryWorkUnit.executeRun(RepositoryWorkUnit.java:93)
at org.talend.core.repository.model.AbstractRepositoryFactory.executeRepositoryWorkUnit(AbstractRepositoryFactory.java:256)
at org.talend.repository.localprovider.model.LocalRepositoryFactory.executeRepositoryWorkUnit(LocalRepositoryFactory.java:3227)
at org.talend.core.repository.model.ProxyRepositoryFactory.executeRepositoryWorkUnit(ProxyRepositoryFactory.java:1996)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction$1.runInUIThread(AbstractCreateTableAction.java:1110)
at org.eclipse.ui.progress.UIJob$1.run(UIJob.java:97)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:136)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3774)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3412)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.talend.rcp.intro.Application.start(Application.java:183)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
at org.eclipse.equinox.launcher.Main.main(Main.java:1438)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.talend.metadata.managment.hive.EmbeddedHiveDataBaseMetadata.getColumns(EmbeddedHiveDataBaseMetadata.java:370)
... 49 more
Caused by: java.lang.RuntimeException: MetaException(message:java.lang.ClassNotFoundException Class com.cloudera.hive.serde.JSONSerDe not found)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:276)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:256)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:595)
at org.apache.hadoop.hive.ql.metadata.Table.getAllCols(Table.java:612)
... 54 more
Caused by: MetaException(message:java.lang.ClassNotFoundException Class com.cloudera.hive.serde.JSONSerDe not found)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:385)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:274)
... 57 more

!ENTRY org.talend.platform.logging 4 0 2015-09-21 10:49:33.620
!MESSAGE 2015-09-21 10:49:33,620 ERROR org.talend.core.model.metadata.builder.database.manager.ExtractManager  - java.lang.RuntimeException: java.sql.SQLException: java.lang.reflect.InvocationTargetException


!ENTRY org.eclipse.ui 4 0 2015-09-21 10:49:33.620
!MESSAGE Unhandled event loop exception
!STACK 0
java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLException: java.lang.reflect.InvocationTargetException
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.returnMetadataColumnsFormTable(ExtractManager.java:525)
at org.talend.core.model.metadata.builder.database.ExtractMetaDataFromDataBase.returnMetadataColumnsFormTable(ExtractMetaDataFromDataBase.java:224)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm.pressRetreiveSchemaButton(DatabaseTableForm.java:1150)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm.access$13(DatabaseTableForm.java:1121)
at org.talend.repository.ui.wizards.metadata.table.database.DatabaseTableForm$4.widgetSelected(DatabaseTableForm.java:795)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4454)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1388)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3799)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3409)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:832)
at org.eclipse.jface.window.Window.open(Window.java:808)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction.handleWizard(AbstractCreateTableAction.java:139)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction$1$1.run(AbstractCreateTableAction.java:1088)
at org.talend.repository.RepositoryWorkUnit.executeRun(RepositoryWorkUnit.java:93)
at org.talend.core.repository.model.AbstractRepositoryFactory.executeRepositoryWorkUnit(AbstractRepositoryFactory.java:256)
at org.talend.repository.localprovider.model.LocalRepositoryFactory.executeRepositoryWorkUnit(LocalRepositoryFactory.java:3227)
at org.talend.core.repository.model.ProxyRepositoryFactory.executeRepositoryWorkUnit(ProxyRepositoryFactory.java:1996)
at org.talend.repository.metadata.ui.actions.metadata.AbstractCreateTableAction$1.runInUIThread(AbstractCreateTableAction.java:1110)
at org.eclipse.ui.progress.UIJob$1.run(UIJob.java:97)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:136)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3774)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3412)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.talend.rcp.intro.Application.start(Application.java:183)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
at org.eclipse.equinox.launcher.Main.main(Main.java:1438)
Caused by: java.lang.RuntimeException: java.sql.SQLException: java.lang.reflect.InvocationTargetException
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.extractColumns(ExtractManager.java:779)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.returnMetadataColumnsFormTable(ExtractManager.java:521)
... 46 more
Caused by: java.sql.SQLException: java.lang.reflect.InvocationTargetException
at org.talend.metadata.managment.hive.EmbeddedHiveDataBaseMetadata.getColumns(EmbeddedHiveDataBaseMetadata.java:401)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.getColumnsResultSet(ExtractManager.java:844)
at org.talend.core.model.metadata.builder.database.manager.ExtractManager.extractColumns(ExtractManager.java:641)
... 47 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.talend.metadata.managment.hive.EmbeddedHiveDataBaseMetadata.getColumns(EmbeddedHiveDataBaseMetadata.java:370)
... 49 more
Caused by: java.lang.RuntimeException: MetaException(message:java.lang.ClassNotFoundException Class com.cloudera.hive.serde.JSONSerDe not found)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:276)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:256)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:595)
at org.apache.hadoop.hive.ql.metadata.Table.getAllCols(Table.java:612)
... 54 more
Caused by: MetaException(message:java.lang.ClassNotFoundException Class com.cloudera.hive.serde.JSONSerDe not found)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:385)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:274)
... 57 more



How can i add my SerDe in Talend Openstudio?


Accepted Solutions
One Star

Re: [resolved] HiveDB Connection needs my own SerDe

Hi 
I solved this problem.
I changed Hive connection settings from Embedded to standalone.
By default standalone connection is trying to read/write to Hive metastore. But all settings points on Hiveserver2. So i got this java.lang.reflect.InvocationTargetException
With standalone  it uses Hiveserver2.

All Replies
One Star

Re: [resolved] HiveDB Connection needs my own SerDe

Hi 
I solved this problem.
I changed Hive connection settings from Embedded to standalone.
By default standalone connection is trying to read/write to Hive metastore. But all settings points on Hiveserver2. So i got this java.lang.reflect.InvocationTargetException
With standalone  it uses Hiveserver2.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now