Four Stars

Dynamic Schema: Loop through all fields - tJavaRow

Does anyone know a way to loop through all fields in a dynamic schema input, using the tJavaRow component?
I'm trying to implement a schema validation process, however the source input data utilizes a dynamic schema, so I'm unable to utilize the tSchemaComplianceCheck component. I want to create a custom routine to determine the data type of each record in each field, however I'm not sure how to implement this routine so that it can be applied to all fields in the schema?
Thanks!

  • Data Integration
11 REPLIES
Community Manager

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi mattirvin
Take a look at my last post in this topic to know how to get the schema properties and the column value.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

Thanks Shong. Most helpful.
Is there a method I can use to get the column data type and column length as defined in dynamic schema definition?
ie:
ColumnType = column.GetType();
ColumnLength = column.GetLength()
I have tried to implement this into the example you provided but when I have an integer field and a string field defined in the dynamic schema, both are returned as "id_String" when I use the GetType method.
Is there another way to get the schema metadata so that I can use it to do data validation checks?
Community Manager

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi
I have tried to implement this into the example you provided but when I have an integer field and a string field defined in the dynamic schema, both are returned as "id_String" when I use the GetType method.

I tested and it works, I have two columns in my table, one type is Integer (11) and another type is varchar(12), the job executes and returns:
Starting job testDynacal at 15:53 10/07/2013.

connecting to socket on port 3546
connected
The length of column is: 11column Type is: id_Integer
The length of column is: 11column Type is: id_String
The length of column is: 11column Type is: id_Integer
The length of column is: 11column Type is: id_String
disconnected
Job testDynacal ended at 15:53 10/07/2013.

Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi Shong, Thank you this is helpful. However I noticed that the length printed in your example is always 11 and it doesn't return 12 for the varchar field.
I have also been having this issue where I am unable to retrieve the correct length using the getLength() method.
Can you confirm if this is a bug?
Community Manager

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi mattirvin
Maybe I posted a wrong result in my previous post, however, I execute the job again today and see it is getting the right result.
tJavaRow:
int ncount=input_row.c.getColumnCount();
DynamicMetadata column0=row1.c.getColumnMetadata(ncount-2);
DynamicMetadata column1=row1.c.getColumnMetadata(ncount-1);
System.out.println("The column name is:"+column0.getName()+" The length of column is: "+column0.getLength()+" The column Type is: "+column0.getType());
System.out.println("The column name is:"+column1.getName()+" The length of column is: "+column1.getLength()+" The column Type is: "+column1.getType());

If you still have doubts, please show me your details.
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi Shong, I think the difference is that I am using a tSetDynamicSchema component to set the Dynamic field first.
I am also using a tJavaFlex component to extract the schema metadata. The issue I'm having is that the getLength() is not returning the correct length as is defined in the input row to the tSetDynamicSchema.
Does it work for you when using the tSetDynamicSchema component?
Please see my job details below:
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

Hi Shong, were you able to replicate this issue on your side using the tSetDynamicSchema component?
Community Manager

Re: Dynamic Schema: Loop through all fields - tJavaRow

Sorry, I have missed your topic, I will investigate your new problem and come back to you asap!
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

Thanks Shong. Any update so far? I have tried numerous times but still can not get the getType or getLength methods to work with tSetDynamicSchema. The getName() does seem to work though.
Four Stars

Re: Dynamic Schema: Loop through all fields - tJavaRow

I found the issue and resolved it.
The code for the variable column in the tJavaFlex component was pointing to the dynamic metadata in the tFileInputDelimited component, instead I pointed to the tSetDynamicSchema component and it works.
The tFileInputDelimited was setting all columns to id_String, 100 no matter the dynamic schema definition.
One Star

Re: Dynamic Schema: Loop through all fields - tJavaRow

hi,
i come on discussion because i have maybe the same problem.
this my csv : id ;name
                 1;kpk
                2; toto
i use dynamic column because my csv structure can change.
this is my workflow: tfileInputDelimited =>tjavaflex=>tmap=tmap;
i arrive to retrieve to tjavaflex all columns from dynamic column of tfileinputdelimited.
so i want to load all columns to tmap  and to do transformation.
but how can i do it.
thank you for your help