Dynamic Schema with Hive

Four Stars

Dynamic Schema with Hive

Hi Everyone,

We are faced with reading a huge number of hive tables with different schemas. So we were hoping we can develop a single talend job to read these different tables. We recently found out that big data components do not support dynamic schema/type. Does anybody know of a possible solution or work around towards changing schemas with big data components?


Re: Dynamic Schema with Hive


Are you in Standard DI job or Bigdata batch job?

Here is a new feature jira issue about "Support Hive Metastore HA".


Best regards


Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: Dynamic Schema with Hive

Hi Sabrina


There's an article "How to operationalize meta-data in Talend with Dynamic Schemas" by Robert Griswold which mentions you can use a String for big data components since Dynamic Schema is not supported by Big Data components.


The blog however doesn't mention how this can be done, do you know a way?



What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.



Introduction to Talend Open Studio for Data Integration.


Downloads and Trials

Test drive Talend's enterprise products.