Error in batch job reading Hive

Five Stars

Error in batch job reading Hive

Hi guys,

 

I am trying to read from Hive via jdbc in a Batch Map Reduce Job.

 

Capture.PNG

However, when I run the job it gives me the following error:

Starting job testmr at 10:40 15/06/2018.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[statistics] connecting to socket on port 3554
[statistics] connected
[WARN ]: org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
Running job: job_1527425158191_0638
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000001_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000000_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 50% reduce 0%
Task Id : attempt_1527425158191_0638_m_000001_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Task Id : attempt_1527425158191_0638_m_000001_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 100% reduce 0%
Job complete: job_1527425158191_0638
Counters: 12
Job Counters
Failed map tasks=7
Killed map tasks=1
Launched map tasks=8
Other local map tasks=8
Total time spent by all maps in occupied slots (ms)=22970
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=22970
Total vcore-milliseconds taken by all map tasks=22970
Total megabyte-milliseconds taken by all map tasks=23521280
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Job Failed: Task failed task_1527425158191_0638_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

java.io.IOException: Job failed!
at org.talend.hadoop.mapred.lib.MRJobClient.runJob(MRJobClient.java:154)
at testproj.testmr_0_1.testmr.runMRJob(testmr.java:1802)
at testproj.testmr_0_1.testmr.access$1(testmr.java:1792)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1390)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at testproj.testmr_0_1.testmr.tJDBCInput_1Process(testmr.java:1341)
at testproj.testmr_0_1.testmr.run(testmr.java:1770)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at testproj.testmr_0_1.testmr.runJobInTOS(testmr.java:1725)
at testproj.testmr_0_1.testmr.main(testmr.java:1673)
[statistics] disconnected
Job testmr ended at 10:41 15/06/2018. [exit code=1]

 

I know that OFFSET is not supported in hive, but I don't have 'OFFSET' in my select statement:

 

Capture2.PNG

 

Do you have any idea how to fix this?

Tags (3)
Moderator

Re: Error in batch job reading Hive

Hello,

Are you able to successfully read data from hive by using talend hive components?

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hi Sabrina,

 

Yes I am able to read data successfully from hive with tHiveInput.

 

Best regards,

Yoanna

Moderator

Re: Error in batch job reading Hive

Hello,

What's hive server version are you trying to connect?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hello,

The version is: Hive 1.2.1000.2.5.0.0-1245 .

Best regards

Moderator

Re: Error in batch job reading Hive

Hello,

Is your query working perfectly fine? Was it some copy/paste issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

As I mentioned earlier I am able to read data with the same query via hive components. Moreover, when I execute the query in hive in hive(via putty) it works. It gives me errors with the jdbc components in a map reduce job.

 

Best regards,

Yoanna

 

 

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads