Exception in thread "main" java.lang.UnsatisfiedLinkError:

One Star

Exception in thread "main" java.lang.UnsatisfiedLinkError:

Hi,
We have TOS BD 6.1.1 installed on Windows 7 64 bit - while trying to write Windos local file to HDFS bu using tHDFSOutput - facing below error. Please help ASAP.
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(II[BI[BIILjava/lang/String;JZ)V
                at org.apache.hadoop.util.NativeCrc32.nativeComputeChunkedSumsByteArray(Native Method)
                at org.apache.hadoop.util.NativeCrc32.calculateChunkedSumsByteArray(NativeCrc32.java:86)
                at org.apache.hadoop.util.DataChecksum.calculateChunkedSums(DataChecksum.java:430)
                at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:202)
                at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
                at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:144)
                at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2250)
                at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
                at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
                at sun.nio.cs.StreamEncoder.implClose(Unknown Source)
                at sun.nio.cs.StreamEncoder.close(Unknown Source)
                at java.io.OutputStreamWriter.close(Unknown Source
at java.io.BufferedWriter.close(Unknown Source)
                at local_project.test_0_1.Test.tFileInputExcel_1Process(Test.java:1308)
                at local_project.test_0_1.Test.tHDFSConnection_1Process(Test.java:374)
                at local_project.test_0_1.Test.runJobInTOS(Test.java:1567)
                at local_project.test_0_1.Test.main(Test.java:1424)
Many Thanks
Regards,
Kiran G.
Moderator

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Hi Kiran G,
Could you please take a look at this jira issue:https://jira.talendforge.org/browse/TBD-2462?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Hi Sabrina,
many thanks for quick reply.
We have gone through the thread unfortunately we don't see any solution Smiley Sad
So this is because of hadoop.dll native library missing or not compatible.
We are connecting to a remote server where Hadoop 2.6.0 CDH5.4 YARN installed - it means we need Windows 7 64 bit supporting Hadoop native binary/library files and same we have to mention in HADOOP_HOME system variable.
If my above understanding is correct, could you please help us with required native library/binary files for windows 64 bit ?
Please suggest what has to be done to get rid of this issue. Many Thanks.
Regards,
Kiran G.
One Star

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Hi ,
I was able to download hadoop.dll and winutils.exe from github for Hadoop 2.6 Windows common. However still facing the same issue. Please suggest.
Kiran G.
Moderator

Re: Exception in thread "main" java.lang.UnsatisfiedLinkError:

Hi,
Could you please try this:
1 - Download http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe and put it in a folder bin folder in your windows machine. For example: C:/tmp/winutils/bin/winutils.exe
2 - In Talend, before executing a job, go in the Advanced Parameters tab of the Run view and check the "Use specific JVM argument". Then, click on new in order to add a new entry:
-> -Dhadoop.home.dir=C:/tmp/winutils
3 - Save and execute your job.
https://jira.talendforge.org/browse/TBD-1412
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.