Five Stars

Text cannot be cast to org.apache.hadoop.hive.ql.io.orc.OrcStruct Error when we try to write HDFS directory in ORC format from talend

Hi,

 

I know this is more of hive query issue and nothing to do with Talend. But, I feel that I can get guidance from this community

We get the Error Text cannot be cast to org.apache.hadoop.hive.ql.io.orc.OrcStruct Error when we try to insert data into hdfs directory in ORC format.

 

The query we use is give below (Just a sample one give masking the original fields and table name)

 

INSERT OVERWRITE DIRECTORY '/mainfolder/subfolder/Table_Name/temp'  stored as ORC  select CAST(Field_1 as string) as Field_1,select CAST(Field_2 as string) as Field_2, Field_3 from database.Table_test_data.

 

Note: there is no CAST on Field_3 as this is int and not varchar.

 

We get this error only when we try with more number of records (million records). but the same query works fine if we run with 1000 records.. Is this data issue ? or is there anything wrong in the query ?

  • Big Data
  • Data Integration