One Star

Pig UDF java.lang.NoSuchMethodError for writing custom UDFs.

Hi TalendForge,
I'm using 5.4.
I can successfully use the UPPER Pig UDF in a tMap like the following:

However, if I make my own Pig UDF using Talend's Eval template, which simply sets a string to uppercase, like:
public class TestUDF extends EvalFunc<String> {
public String exec(Tuple input) throws IOException {
if (input == null || input.size() == 0) {
return null;
try {
String str = (String) input.get(0);
return str.toUpperCase();
} catch (Exception e) {
throw new IOException("Caught exception processing input row ", e);

And change the tMap to use pigudf.TestUDF(row1.newColumn):

I then get the following error:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(
at org.apache.pig.PigServer.launchPlan(
at org.apache.pig.PigServer.executeCompiledLogicalPlan(
at org.apache.pig.PigServer.execute(
at org.apache.pig.PigServer.executeBatch(
at primehome.pigudf2_0_1.PigUDF2.tPigLoad_1Process(
at primehome.pigudf2_0_1.PigUDF2.runJobInTOS(
at primehome.pigudf2_0_1.PigUDF2.main(
Job PigUDF2 ended at 13:29 04/11/2014.

In fact, if I change the UDF back to UPPER which originally worked, the job will still fail with the same error. I have to create a new job to get rid of this error.

Re: Pig UDF java.lang.NoSuchMethodError for writing custom UDFs.

Can you please create a JIRA issue in our issue tracker ?
You can create it in the TBD project.
Thanks in advance,

Re: Pig UDF java.lang.NoSuchMethodError for writing custom UDFs.

It seems this problem comes from a classloading issue. If you right click on the UDF you created in the repository, you can see the UDF dependencies. By default, Talend uses pig-0.10.jar in order to compile the UDF. The problem is that the cluster you work on uses a more recent version of Pig. You should update it with the correct Pig version.