One Star

[resolved] How to create table schema dynamically?

Hi have 4 tables in oracle. i am writing a single job that will pull data from oracle tables and dump to hdfs (instead of writing 4 different jobs for 4 different tables).
I am taking the table name from context.
But while pulling the data from oracle i have to statically specify the schema. This is killing my dynamic objective.
Here is the graph i am using:

In tOracleInput i need the "guess schema" part to be dynamic to guess the schemas for all the tables on the fly. Also in this component tOracleInput i have defined the schema with a single column entry with type "Dynamic".
tConvertType is being used to convert the "Dynamic" type to "String" type.
tJavaRow is being used for trimming the data.
and finally dumping the data into hdfs.
This whole setup is not working. the hdfs output file is containing 0 data. It isnt throwing any error though.
Where am i going wrong ?

1 ACCEPTED SOLUTION

Accepted Solutions
One Star

Re: [resolved] How to create table schema dynamically?

found a solution.
i removed the tConvertType componet.
so now only 3 components:
tOracleInput--->tJavaRow--->tHdfsOutput
In the tJavaRow i added the code:
String row = input_row.newColumn.toString();
3 REPLIES
One Star

Re: [resolved] How to create table schema dynamically?

found a solution.
i removed the tConvertType componet.
so now only 3 components:
tOracleInput--->tJavaRow--->tHdfsOutput
In the tJavaRow i added the code:
String row = input_row.newColumn.toString();
Moderator

Re: [resolved] How to create table schema dynamically?

Hi,

Thanks for posting that you resolved it by yourself. Feel free to let us know if there is any further help we can give.

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: [resolved] How to create table schema dynamically?

Hello friend,

 

I have tried to use the below option which you have mentioned. But its loading all the values as "null" in hdfs file. Is there any way to get the source data instead of null?

tOracleInput--->tJavaRow--->tHdfsOutput

String row = input_row.newColumn.toString();