Talend job not submitting to the defined YARN queue

Four Stars

Talend job not submitting to the defined YARN queue

Hi Experts,

 

We wanted to prioritize some of the talend jobs. hence we setup a dedicated YARN queue 'root.talend.high' and assigned resources to it.

We have also set the paramter as queueName: 'root.talend.high' in talend job.

However, when submitting the jobs it is always landing on 'root.default' queue as show in below logs.

 

Am I missing something here ? need your inputs in fixing this. Thanks in advance 

 

[INFO ]: org.apache.spark.deploy.yarn.Client - Application report for application_1558146131347_298872 (state: ACCEPTED)
[INFO ]: org.apache.spark.deploy.yarn.Client -
client token: Token { kind: YARN_CLIENT_TOKEN, service: }
diagnostics: N/A
ApplicationMaster host: N/A
ApplicationMaster RPC port: -1
queue: root.default
start time: 1561621900134
final status: UNDEFINED

Moderator

Re: Talend job not submitting to the defined YARN queue

Hello,

There are no Error and WARN from your log. Are your job able to stop automatically? Could you please upload your job setting screenshot into community which will be helpful for us to address your issue.

Please mask your sensitive data.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Talend job not submitting to the defined YARN queue

Hi xdshi,

 

Thanks for your response.

 

There is no such error or warning here,job is completed successfully.

 

Only thing is,it is landing in "root.default" YARN queue although I have set the "queueName" to "root.talend.high".

 

Let me modify my question here as I have got some of my answers:

 

Is it possible to change the queueName for a deployed job from TAC console .

(I already verified that it is going in right queue if I update it from Talend Studio)

 

Thanks in advance!

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog