Dynamic Schema with Hive

Four Stars

Dynamic Schema with Hive

Hi Everyone,

We are faced with reading a huge number of hive tables with different schemas. So we were hoping we can develop a single talend job to read these different tables. We recently found out that big data components do not support dynamic schema/type. Does anybody know of a possible solution or work around towards changing schemas with big data components?


Re: Dynamic Schema with Hive


Are you in Standard DI job or Bigdata batch job?

Here is a new feature jira issue about "Support Hive Metastore HA".


Best regards


Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Dynamic Schema with Hive

Hi Sabrina


There's an article "How to operationalize meta-data in Talend with Dynamic Schemas" by Robert Griswold which mentions you can use a String for big data components since Dynamic Schema is not supported by Big Data components.


The blog however doesn't mention how this can be done, do you know a way?