Dynamic Partition for Hive with TOS

Highlighted
Five Stars

Dynamic Partition for Hive with TOS

Hello Experts,

 

As per the requirement i have to load the data in Hive partition table, Talend works fine static partition and however I'm facing issue for Dynamic partition.

 

For EX:

I Created a table with 5 columns like id, name , occupation,zip code and gender(Partition column) , while loading the data if i use gender = 'M' or 'F' in tHiveLoad / thiverow component its working fine, if i'm not passing any value to gender column then i'm getting the error.

 

I am getting the below error, to resolve this i tried with  "hive.exec.dynamic.partition.mode=nonstrict" and "hive.exec.dynamic.partition = true" , but no luck

 

However i'm able to achieve Dynamic partitioning in Hive scripting with out passing any value to partition column, but not in Talend

 

Error while compiling statement: FAILED: SemanticException [Error 10096]: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict

 

Your help is highly appreciate!!

 

Thanks,

Bharath 

 

 

Five Stars

Re: Dynamic Partition for Hive with TOS

Hello Talend Folks,

 

Can anyone please help me on this Dynamic partition?

 

Regards,

Bharath

Moderator

Re: Dynamic Partition for Hive with TOS

Hello,

On which build version you got this issue?

Have you tried to put

hiveContext.setConf("hive.exec.dynamic.partition", "true")
hiveContext.setConf("hive.exec.dynamic.partition.mode", "nonstrict")?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now