Hi, I have a problem running a Spark batch job on MapR.
I'm currently working on a Spark batch job on a cluster with multiple nodes.
But when I ran the job, the job fails, with this message as an output:
"Yarn application has already ended! It might have been killed or unable to launch application master."
I checked the resource manager log and saw that the job failed with this message:
"Requested user root is not whitelisted and has id 0,which is below the minimum allowed 500."
I think the job fails when it runs as user "root". So is there any way I can run the job as a user besides root?
We have a jira issue:https://jira.talendforge.org/browse/TBD-3955 about this error: "Requested user root is not whitelisted and has id 0,which is below the minimum allowed 500".
Let us know if the solution works for you.
Thanks for responding to the problem.
If I understand correctly, the solution to the Jira issue was to use a JobServer.
But I can't find a JobServer directory in my Talend installation directory. I've only managed to find several jars which relate to JobServer.
From this link (https://help.talend.com/reader/ozfCR4oo4noN8gSKT7qP_w/TgjvBs4NXECwoSePsMB0HQ), it seems that there should be a folder named JobServer.
Do I have to do a separate installation in order to use it? Or is the JobServer directory stored somewhere else?
Are you using talend data platform product? Free trial version? Is there any archive file name for example reads: Talend-JobServer-YYYYMMDD_HHmm-VA.B.C.zip?
With you licensed version of Talend Data Fabric, could you please create a case on talend support portal so that we can give you a remote assistance(webex) to see if there is any problem for your job server installation through support cycle with priority?
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to make your data more available, reduce costs and cut your build time
Read about OTTO's experiences with Big Data and Personalized Experiences
Take a look at this video about Talend Integration with Databricks