One Star

Hadoop Cluster connection in Talend 5.5

Hi,
We are trying to connect to Hadoop "PivotalHD 2.0" from Talend 5.5. We can make the connection very well using a standard job components like tHDFSGet or tHDFSList using Kerberos authentication.
But we are facing issues using same connection setting when trying with Hadoop Cluster connection in a Map/Reduce job.
I see the Kerberos authentication is not enabled for PivotalHD 2.0, is this a limitation or we can get this working.
Please suggest.
thanks,
Rahul
GE Aviation
3 REPLIES
Moderator

Re: Hadoop Cluster connection in Talend 5.5

Hi RahulV,

Thanks for posting that you resolved it by yourself. Could you please share your solution with us?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Hadoop Cluster connection in Talend 5.5

We had the Namenode URI wrong. Default by talend is nn_/_HOST, we used hdfs/_HOST and its working to authenticate normal Standard job connection to HDFS.
I have another issue now where when trying to connect to HDFS from a talend mapreduce job, its using SIMPLE authentication even though we have sepcified Kerberos authentication. 
14/10/24 15:22:59 ERROR security.UserGroupInformation: PriviledgedActionException as:<UserName> (authSmiley FrustratedIMPLE) causeSmiley Surprisedrg.apache.hadoop.security.AccessControlException: Client cannot authenticate via:
Error message is above.
Thanks,
Rahul
One Star

Re: Hadoop Cluster connection in Talend 5.5

Hello Sabrina, 
Can you please help with the error we are getting for Mapreduce jobs in connecting to HDFS using Kerberos.
Thanks
Rahul