Mapr Connection

One Star

Mapr Connection

Hi Team ,
I have created the job which have jdbcinput and writes data to hdfs file on mapr system . 

While running the job I am getting error . Can u tell me how to point to mapr client path . 
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See  for more info.
==========Unable to find library on native path due to Exception. ==============
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more
==========Unable to find library in jar due to exception. ==============
java.lang.RuntimeException: no native library is found for os.name=Linux and os.arch=x86_64
        at com.mapr.fs.ShimLoader.findNativeLibrary(ShimLoader.java:496)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:318)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more
Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:214)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        ... 20 more
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more
Moderator

Re: Mapr Connection

Hi,

MapR requires their client library to be in the Path. 
Could you please take a look at document about: http://doc.mapr.com/display/MapR/Setting+Up+the+Client
In the client jar, you will find a DLL (if you are using Windows) in the lib folder. This DLL has to be in the path of windows.

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Mapr Connection

Hi Team ,
I have created the job which have jdbcinput and writes data to hdfs file on mapr system . 

While running the job I am getting error . Can u tell me how to point to mapr client path . 
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See  for more info.
==========Unable to find library on native path due to Exception. ==============
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more
==========Unable to find library in jar due to exception. ==============
java.lang.RuntimeException: no native library is found for os.name=Linux and os.arch=x86_64
        at com.mapr.fs.ShimLoader.findNativeLibrary(ShimLoader.java:496)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:318)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more
Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:214)
        at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:60)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:278)
        at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
        at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
        at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:392)
        at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:375)
        at org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:392)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:288)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at demo.test2_0_1.test2.runJobInTOS(test2.java:1566)
        at demo.test2_0_1.test2.main(test2.java:1551)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at com.mapr.fs.ShimLoader.loadNativeLibrary(ShimLoader.java:308)
        at com.mapr.fs.ShimLoader.load(ShimLoader.java:197)
        ... 20 more
Caused by: java.lang.UnsatisfiedLinkError: no MapRClient in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
        at java.lang.Runtime.loadLibrary0(Runtime.java:849)
        at java.lang.System.loadLibrary(System.java:1088)
        at com.mapr.fs.shim.LibraryLoader.loadLibrary(LibraryLoader.java:41)
        ... 26 more

Have you solved this issue?
Seven Stars

Re: Mapr Connection

Hi,

MapR requires their client library to be in the Path. 
Could you please take a look at document about: http://doc.mapr.com/display/MapR/Setting+Up+the+Client
In the client jar, you will find a DLL (if you are using Windows) in the lib folder. This DLL has to be in the path of windows.

Best regards
Sabrina

I have the same problem while deploying my talend job in linux. What should i do to fix this error?