tPigLoad cannot load any file from HDFS, it is getting the null value

One Star

tPigLoad cannot load any file from HDFS, it is getting the null value

Hi, Guys
I am using your talend big data open studio to testing for big data, and running into an issue
When I tried to use tPigLoad to load file in HDFS, it gives the error please see the screen shot.
I have tried every solution in google, but none does work.

this is my testing schema in hdfs

please help me. Thank you!

I don't know why the screen shot is missed
please see the error
Starting job T_Pigload_Test at 03:08 10/08/2015.
connecting to socket on port 3587
connected
connecting to socket on port 4700
connected
: org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
objc: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_11.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
: org.apache.hadoop.mapreduce.JobSubmitter - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
: org.apache.hadoop.mapreduce.JobSubmitter - No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
: org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed!
disconnected
disconnected
Job T_Pigload_Test ended at 03:09 10/08/2015.


One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

FYI, in the document from talend open big studio, I can successfully run the first 3 example
A_HCatalog_Create
B_HCatalog_Load
C_HCatalog_Read
I have no issue about connect to HCatalog, HDFS and namenode. I setup my own vm cluster and can run the pig command inside the pig shell (grunt) in the cluster.
Just cannot trigger the tPigLoad component in talend to the cluster, I believe the connection to namenode from talend to hadoop cluster is good. Can someone help me?
Thank you so much.
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

I'm having the same issue - same stack trace.
I don't seem able to post screenshots, but my flow is essentially:
tRowGenerator -> tPigLoad --> tPigCode --> tPigStoreResult
If I disable the StoreResult component, I get no errors...
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi 
Were you able to get this problem resolved? I am facing the same issue. Please let me know how you resolved the issue 
Thanks
Krithika
Moderator

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi Krithika,
On which build version you got this issue? What's your work flow?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Talend version 6.1.1
TPigload- tpigstoreresult
Reading a file from hdfs in mapreduce mode and storing the results in a different directory in hdfs
Can you please help? It is urgentt
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Can anyone pls respond to the questions above?
Moderator

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi kswamina,
Are you able to connect to HDFS? What's your OS and hadoop version?
Is your Talend Studio Java version is the same as the cluster? 1.8? More information will be preferred.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi Sabrina,
Yes, I am able to connect to HDFS. I am using windows 7 enterprise. Hadoop version is 2.7.1. Java version 1.8.0_65 in both talend and cluster.

Namenode URI "hdfs://10.56.70.93:9000"
Resource Manager "10.56.70.93:8032"
Resource Manager Scheduler Address: "10.56.70.93:8030"
If I remove the tPigStoreResult component from the job, i don't get any error.
Please helpp. If you need more information, please let me know
thanks
Krithika
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi Sabrina,
Also, please find attached the screenshot for the above issue
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi, 
When i export the job and run it as shell script, this is the error message i get.
6/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: job job_1454362501785_0008 has failed! Stop running all dependent jobs
6/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: 100% complete
6/02/01 20:25:13 INFO client.RMProxy: Connecting to ResourceManager at /10.56.70.93:8032
6/02/01 20:25:13 INFO client.RMProxy: Connecting to ResourceManager at /10.56.70.93:8032
6/02/01 20:25:13 ERROR mapreduce.MRPigStatsUtil: 1 map reduce job(s) failed!
6/02/01 20:25:13 INFO mapreduce.SimplePigStats: Script Statistics: 
HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures
2.7.1.2.3.2.0-29500.15.0.2.3.2.0-2950skrithika2016-02-01 20:25:072016-02-01 20:25:13UNKNOWN
Failed!
Failed Jobs:
JobIdAliasFeatureMessageOutputs
job_1454362501785_0008tPigLoad_1_row1_RESULTMAP_ONLYMessage: Job failed!/user/skrithika/pigstore,
Input(s):
Failed to read data from "/user/skrithika/test_data/data.txt"
Output(s):
Failed to produce result in "/user/skrithika/pigstore"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_1454362501785_0008

16/02/01 20:25:13 INFO mapReduceLayer.MapReduceLauncher: Failed!
Moderator

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Hi,
Have you already checked document:TalendHelpCenterSmiley Frustratedupported Hadoop distribution versions to see if you are on supported Hadoop distribution?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

Yes, it is supported. Checked. This could be a yarn issue? 
One Star

Re: tPigLoad cannot load any file from HDFS, it is getting the null value

it looks more like a code generation issue.