Six Stars

tFileOutputDelimited not using queue for mapreduce process

Hi, I have a job that extracts the XML files from Hbase in Hadoop and parse it to the pipe delimited text format file. So my question here is the MapReduce-queue is not used for this job although I see  the pipe delimited data before completion of the data. Is there a case this happens in talend big data studio? Please let me know if any idea to this.

My question is  

1 ACCEPTED SOLUTION

Accepted Solutions
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

This is because job running on standalone mode. I need to configure in yarn mode for queue.

9 REPLIES
Moderator

Re: tFileOutputDelimited not using queue for mapreduce process

Hello,

Could you please post your current job setting screenshots on forum which will be helpful for us to address your issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

sure. Attaching along to this.. talend job screenshot.JPG

Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

I posted the screenshot. Any idea to this? Anyone?? Any idea??

Moderator

Re: tFileOutputDelimited not using queue for mapreduce process

Hello,

Do you mean not all messages are correctly transferred from tfileoutputdelimited_1-->tHDFSOutput?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

No. Messages are correctly transferred however I don't see logs on Resource Manager on this. When we are extracting XML files and parsing/converting to Text (Pipe Delimited or other), there should be MapReduce process taken place for this. But I don't see this happening.
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

So no solution to this?

Moderator

Re: tFileOutputDelimited not using queue for mapreduce process

Hi,

We cannot repro this issue on our side. Could you please indicate on which build version you got this issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

6.3 version.
Six Stars

Re: tFileOutputDelimited not using queue for mapreduce process

This is because job running on standalone mode. I need to configure in yarn mode for queue.