One Star

Hadoop Cluster connection in Talend 5.5

We are trying to connect to Hadoop "PivotalHD 2.0" from Talend 5.5. We can make the connection very well using a standard job components like tHDFSGet or tHDFSList using Kerberos authentication.
But we are facing issues using same connection setting when trying with Hadoop Cluster connection in a Map/Reduce job.
I see the Kerberos authentication is not enabled for PivotalHD 2.0, is this a limitation or we can get this working.
Please suggest.
GE Aviation

Re: Hadoop Cluster connection in Talend 5.5

Hi RahulV,

Thanks for posting that you resolved it by yourself. Could you please share your solution with us?

Best regards
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Hadoop Cluster connection in Talend 5.5

We had the Namenode URI wrong. Default by talend is nn_/_HOST, we used hdfs/_HOST and its working to authenticate normal Standard job connection to HDFS.
I have another issue now where when trying to connect to HDFS from a talend mapreduce job, its using SIMPLE authentication even though we have sepcified Kerberos authentication. 
14/10/24 15:22:59 ERROR security.UserGroupInformation: PriviledgedActionException as:<UserName> (authSmiley FrustratedIMPLE) causeSmiley Client cannot authenticate via:
Error message is above.
One Star

Re: Hadoop Cluster connection in Talend 5.5

Hello Sabrina, 
Can you please help with the error we are getting for Mapreduce jobs in connecting to HDFS using Kerberos.