Dynamic Schema

Highlighted
Five Stars

Dynamic Schema

I am evaluating Talnd Big Data edition for data consumption job.
I have a set of 5 tables , my requirements is join these five tables and generate subset of columns as o/p in a hive table.
Number of Output columns are not fixed , that is why I need to generate schema dynamically.
I am using tHiveInput to read the data and tsqrow to join these five tables.
Questions -
1. Can tSqlRow generate schema dynamically ?
2. I can use tjava but the problem with t Java is it can't take more than one input.
3. I can't find tSetDynamic component in talend big data studio.
Thanks




Tags (1)
Employee

Re: Dynamic Schema

Hi,

 

    I would recommend to go through the below blogs.

 

https://www.talend.com/blog/2019/10/31/magically-shuffle-columns-talend-dynamic-schema/

https://www.talend.com/blog/2019/11/11/migrate-data-between-databases-one-job-using-dynamic-schema/

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

 

Five Stars

Re: Dynamic Schema

Thanks for the response. 

However the links are not applicable to Talend Big Data Jobs.

Talend Big Data Job does not support dynamic schema which is a major limitation. Is there a workaround of this limitation ?

if we cant overcome this limitatatin we cant really use talend , in that case we need to write spark code in pure java 

 

Thanks

 

 

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now