From Thursday, July, 9, 3:00 PM Pacific,
our Community site will be in
read-only mode
through Sunday, July 12th.
Thank you for your patience.

Adding SerDe Jar in Hive with tHiveInput

Highlighted
One Star

Adding SerDe Jar in Hive with tHiveInput

We have a table in Hive that requires a SerDe to parse the data. In other to run the query to extract the data in Talend, we need to add the SerDe Jar files prior to the SQL statement.
Using TOS Big Data v5.3, with tHiveInput, I was not able to add both 'ADD JAR /usr/lib/hive/lib/contrib.jar;' and the 'select' statement because it won't allow multiple SQL statements within the query.
The current work around is to create another physical table in Hive with create-table-as outside of Talend.
Are there other ways?
Highlighted

Re: Adding SerDe Jar in Hive with tHiveInput

Hi
would tLibraryLoad work?
Regards,
Brandon
Highlighted
One Star

Re: Adding SerDe Jar in Hive with tHiveInput

Thanks Brandon for responding. I tried the tLibraryLoad, but it seemed like the SerDe Jar needed to be added within the Hive session.
We are also playing around the idea of changing the hive-site.xml to include the contrib.jar by default.
Highlighted
Employee

Re: Adding SerDe Jar in Hive with tHiveInput

Hello,
I think you need to do additional things in order to add the JAR in the Hive classpath.
ADD JAR ?s3n://matthewsbucket/superudf.jar?;
CREATE TEMPORARY FUNCTION super as ?com.matthewrathbone.SuperFunction?;
You could try to use that with two tHiveRow components.
Regards,
Rémy.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now