Four Stars

Hadoop Cluster Conn - Hadoop Properties to Context problems

Hi,
I'm using TOS BD version 6.0.1.  I set up a Metadata Hadoop ClusterConnection with some Hadoop properties specified to support HA. 
I then exported as a context.  All that seems to work fine.  The context definition looks correct except for a couple problems.
1. Some hadoop properties have a "-" (hyphen) in the name like "dfs.ha.automatic-failover.enabled.epsdatalake".  When these properties get exported to a context they automatically get a name like context.ClusterName_dfs_ha_automatic-failover_enabled_epsdatalake.  The "-" (hyphen) is illegal for Java variable names and will cause compiler errors when it's used.  This is easy enough to work around by manually editing context variable name.
2. When creating a component with the Repository connection, like tHDFSConnection the Hadoop properties do automatically show up in the Hadoop Properties table as one would expect.  However the all the context variables in the Value column are quoted.  They are context variables at this point and not text so should not be quoted.  Removing the quotes allows jobs to work correctly, however then the connection Property Type changes from Repository to Built-In.  This negates much of the point of using a repository connection.
 Thanks
Pete
1 REPLY
Moderator

Re: Hadoop Cluster Conn - Hadoop Properties to Context problems

Hi Pete,
When creating a component with the Repository connection, like tHDFSConnection the Hadoop properties do automatically show up in the Hadoop Properties table as one would expect.  However the all the context variables in the Value column are quoted.  They are context variables at this point and not text so should not be quoted.  Removing the quotes allows jobs to work correctly, however then the connection Property Type changes from Repository to Built-In.  This negates much of the point of using a repository connection.

Would you mind posting your tHDFSConnection component setting screenshot with the context variables into forum?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.