[resolved] How to create table schema dynamically?

One Star

[resolved] How to create table schema dynamically?

Hi have 4 tables in oracle. i am writing a single job that will pull data from oracle tables and dump to hdfs (instead of writing 4 different jobs for 4 different tables).
I am taking the table name from context.
But while pulling the data from oracle i have to statically specify the schema. This is killing my dynamic objective.
Here is the graph i am using:

In tOracleInput i need the "guess schema" part to be dynamic to guess the schemas for all the tables on the fly. Also in this component tOracleInput i have defined the schema with a single column entry with type "Dynamic".
tConvertType is being used to convert the "Dynamic" type to "String" type.
tJavaRow is being used for trimming the data.
and finally dumping the data into hdfs.
This whole setup is not working. the hdfs output file is containing 0 data. It isnt throwing any error though.
Where am i going wrong ?


Accepted Solutions
Highlighted
One Star

Re: [resolved] How to create table schema dynamically?

found a solution.
i removed the tConvertType componet.
so now only 3 components:
tOracleInput--->tJavaRow--->tHdfsOutput
In the tJavaRow i added the code:
String row = input_row.newColumn.toString();

View solution in original post


All Replies
Highlighted
One Star

Re: [resolved] How to create table schema dynamically?

found a solution.
i removed the tConvertType componet.
so now only 3 components:
tOracleInput--->tJavaRow--->tHdfsOutput
In the tJavaRow i added the code:
String row = input_row.newColumn.toString();

View solution in original post

Moderator

Re: [resolved] How to create table schema dynamically?

Hi,

Thanks for posting that you resolved it by yourself. Feel free to let us know if there is any further help we can give.

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: [resolved] How to create table schema dynamically?

Hello friend,

 

I have tried to use the below option which you have mentioned. But its loading all the values as "null" in hdfs file. Is there any way to get the source data instead of null?

tOracleInput--->tJavaRow--->tHdfsOutput

String row = input_row.newColumn.toString();

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now