Hadoop Cluster connection in Talend 5.5

One Star

Hadoop Cluster connection in Talend 5.5

Hi,
We are trying to connect to Hadoop "PivotalHD 2.0" from Talend 5.5. We can make the connection very well using a standard job components like tHDFSGet or tHDFSList using Kerberos authentication.
But we are facing issues using same connection setting when trying with Hadoop Cluster connection in a Map/Reduce job.
I see the Kerberos authentication is not enabled for PivotalHD 2.0, is this a limitation or we can get this working.
Please suggest.
thanks,
Rahul
GE Aviation
Moderator

Re: Hadoop Cluster connection in Talend 5.5

Hi RahulV,

Thanks for posting that you resolved it by yourself. Could you please share your solution with us?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Hadoop Cluster connection in Talend 5.5

We had the Namenode URI wrong. Default by talend is nn_/_HOST, we used hdfs/_HOST and its working to authenticate normal Standard job connection to HDFS.
I have another issue now where when trying to connect to HDFS from a talend mapreduce job, its using SIMPLE authentication even though we have sepcified Kerberos authentication. 
14/10/24 15:22:59 ERROR security.UserGroupInformation: PriviledgedActionException as:<UserName> (authSmiley FrustratedIMPLE) causeSmiley Surprisedrg.apache.hadoop.security.AccessControlException: Client cannot authenticate via:
Error message is above.
Thanks,
Rahul
One Star

Re: Hadoop Cluster connection in Talend 5.5

Hello Sabrina, 
Can you please help with the error we are getting for Mapreduce jobs in connecting to HDFS using Kerberos.
Thanks
Rahul