Error in batch job reading Hive

Highlighted
Five Stars

Error in batch job reading Hive

Hi guys,

 

I am trying to read from Hive via jdbc in a Batch Map Reduce Job.

 

Capture.PNG

However, when I run the job it gives me the following error:

Starting job testmr at 10:40 15/06/2018.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[statistics] connecting to socket on port 3554
[statistics] connected
[WARN ]: org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
Running job: job_1527425158191_0638
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000001_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000000_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 50% reduce 0%
Task Id : attempt_1527425158191_0638_m_000001_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Task Id : attempt_1527425158191_0638_m_000001_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 100% reduce 0%
Job complete: job_1527425158191_0638
Counters: 12
Job Counters
Failed map tasks=7
Killed map tasks=1
Launched map tasks=8
Other local map tasks=8
Total time spent by all maps in occupied slots (ms)=22970
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=22970
Total vcore-milliseconds taken by all map tasks=22970
Total megabyte-milliseconds taken by all map tasks=23521280
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Job Failed: Task failed task_1527425158191_0638_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

java.io.IOException: Job failed!
at org.talend.hadoop.mapred.lib.MRJobClient.runJob(MRJobClient.java:154)
at testproj.testmr_0_1.testmr.runMRJob(testmr.java:1802)
at testproj.testmr_0_1.testmr.access$1(testmr.java:1792)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1390)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at testproj.testmr_0_1.testmr.tJDBCInput_1Process(testmr.java:1341)
at testproj.testmr_0_1.testmr.run(testmr.java:1770)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at testproj.testmr_0_1.testmr.runJobInTOS(testmr.java:1725)
at testproj.testmr_0_1.testmr.main(testmr.java:1673)
[statistics] disconnected
Job testmr ended at 10:41 15/06/2018. [exit code=1]

 

I know that OFFSET is not supported in hive, but I don't have 'OFFSET' in my select statement:

 

Capture2.PNG

 

Do you have any idea how to fix this?

Moderator

Re: Error in batch job reading Hive

Hello,

Are you able to successfully read data from hive by using talend hive components?

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hi Sabrina,

 

Yes I am able to read data successfully from hive with tHiveInput.

 

Best regards,

Yoanna

Moderator

Re: Error in batch job reading Hive

Hello,

What's hive server version are you trying to connect?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hello,

The version is: Hive 1.2.1000.2.5.0.0-1245 .

Best regards

Moderator

Re: Error in batch job reading Hive

Hello,

Is your query working perfectly fine? Was it some copy/paste issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

As I mentioned earlier I am able to read data with the same query via hive components. Moreover, when I execute the query in hive in hive(via putty) it works. It gives me errors with the jdbc components in a map reduce job.

 

Best regards,

Yoanna

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now