Error in batch job reading Hive

Five Stars

Error in batch job reading Hive

Hi guys,

 

I am trying to read from Hive via jdbc in a Batch Map Reduce Job.

 

Capture.PNG

However, when I run the job it gives me the following error:

Starting job testmr at 10:40 15/06/2018.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/Yoanna.Vasileva/workspace/.Java/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
[WARN ]: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[statistics] connecting to socket on port 3554
[statistics] connected
[WARN ]: org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
Running job: job_1527425158191_0638
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000001_0, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
Task Id : attempt_1527425158191_0638_m_000000_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 50% reduce 0%
Task Id : attempt_1527425158191_0638_m_000001_1, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 0% reduce 0%
Task Id : attempt_1527425158191_0638_m_000000_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Task Id : attempt_1527425158191_0638_m_000001_2, Status : FAILED
Error: java.io.IOException: Error while compiling statement: FAILED: ParseException line 1:192 missing EOF at 'OFFSET' near '139428'
at testproj.testmr_0_1.testmr$row2StructInputFormat.getRecordReader(testmr.java:611)
at org.talend.hadoop.mapred.lib.DelegatingInputFormat.getRecordReader(DelegatingInputFormat.java:59)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
map 100% reduce 0%
Job complete: job_1527425158191_0638
Counters: 12
Job Counters
Failed map tasks=7
Killed map tasks=1
Launched map tasks=8
Other local map tasks=8
Total time spent by all maps in occupied slots (ms)=22970
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=22970
Total vcore-milliseconds taken by all map tasks=22970
Total megabyte-milliseconds taken by all map tasks=23521280
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
Job Failed: Task failed task_1527425158191_0638_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

java.io.IOException: Job failed!
at org.talend.hadoop.mapred.lib.MRJobClient.runJob(MRJobClient.java:154)
at testproj.testmr_0_1.testmr.runMRJob(testmr.java:1802)
at testproj.testmr_0_1.testmr.access$1(testmr.java:1792)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1390)
at testproj.testmr_0_1.testmr$1.run(testmr.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at testproj.testmr_0_1.testmr.tJDBCInput_1Process(testmr.java:1341)
at testproj.testmr_0_1.testmr.run(testmr.java:1770)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at testproj.testmr_0_1.testmr.runJobInTOS(testmr.java:1725)
at testproj.testmr_0_1.testmr.main(testmr.java:1673)
[statistics] disconnected
Job testmr ended at 10:41 15/06/2018. [exit code=1]

 

I know that OFFSET is not supported in hive, but I don't have 'OFFSET' in my select statement:

 

Capture2.PNG

 

Do you have any idea how to fix this?

Tags (3)
Moderator

Re: Error in batch job reading Hive

Hello,

Are you able to successfully read data from hive by using talend hive components?

Best regards

Sabrina

 

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hi Sabrina,

 

Yes I am able to read data successfully from hive with tHiveInput.

 

Best regards,

Yoanna

Moderator

Re: Error in batch job reading Hive

Hello,

What's hive server version are you trying to connect?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

Hello,

The version is: Hive 1.2.1000.2.5.0.0-1245 .

Best regards

Moderator

Re: Error in batch job reading Hive

Hello,

Is your query working perfectly fine? Was it some copy/paste issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Error in batch job reading Hive

As I mentioned earlier I am able to read data with the same query via hive components. Moreover, when I execute the query in hive in hive(via putty) it works. It gives me errors with the jdbc components in a map reduce job.

 

Best regards,

Yoanna