One Star

No password in HDFSconnection

Hi all,
I successfully connected a tHDFSconnection component to upload files into my HDFS (a HDP cluster). But I'm very surprised that the component is asking for a user name but not for password. More than that, without specifying any password, I can use any user name (even hdfs the super user).
How do you explain this ? Should I active some security configuration in HDFS ? In this case, how am I suppose to send the password from talend ?
Do you have any documentation on that?
I'm not sure if I should post this question here, but since I have this concern using Talend Big Data, maybe one of you guys could eventually help me Smiley Happy
Thank you in advance anyway.
Regards,
Orlando
5 REPLIES
Moderator

Re: No password in HDFSconnection

Hi Orlando,
Have you checked the "Authentication" with using kerberos authentication and a keytab to authentication?
Please have a look at component reference:TalendHelpCenter:tHDFSConnection
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: No password in HDFSconnection

Thank you for you help.
I thought that a login AND password was required even with a basic authentication, without using Kerberos.
I'll enable Kerberos in this case.
Thanks again for you reactivity.
Regards,
Orlando
One Star

Re: No password in HDFSconnection

Hi again,
to address my basic security needs, I was planing to go through a gateway with Apache Knox and rely on the network firewalls.
But I am still wondering how I would pass my password from talend ? Will I need to write userSmiley Tongueassword in the user field of the tHDFSconnection ?
Do you have any documentation on using knox together with talend ?
Thanks.
Bests,
Orlando
One Star

Re: No password in HDFSconnection

We are getting the below error for tHDFSConnection, need to know where we need to pass the password for establishing connection.
Exception in component tHDFSConnection_1
java.io.IOException: Login failure for hdfs from keytab /tmp/hdfs.headless.
at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:592)
at projectname.hadoopwrite_0_1.HadoopWrite.tHDFSConnection_1Process(HadoopWrite.java:347)
at projectname.hadoopwrite_0_1.HadoopWrite.runJobInTOS(HadoopWrite.java:1350)
at projectname.hadoopwrite_0_1.HadoopWrite.main(HadoopWrite.java:1207)
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user
One Star

Re: No password in HDFSconnection

Hi All
Do you have a solution connecting Talend to Hadoop where Hadoop is secured with Knox only (no Kerberos) for HDFS, HIVE, OOZIE, HBASE