Four Stars

How can I set mapreduce/hive job parameters in a tHiveInput query?

I am trying to export the output of the following query . Before that I want to set some hive properties.

SET hive.execution.engine=tez;

SET hive.tez.container.size=8192;

set hive.merge.mapfiles=true;

set hive.merge.mapredfiles=true;

set hive.merge.tezfiles=true;

 

I am building the query using various context variables.

"select tbl.hd from (select tab1.hd, tab1.cl1 from (" + context.getProperty("CUSTOM_SEL") + context.getProperty("HDR_COL) + context.getProperty("CUST_HEADER") + context.getProperty("CUST_UNION") + context.SQL_TEXT+") tab1 order by tab1.cl1 asc ) tbl"

Is there way I can execute the SET hive parameters before executing select query ?

2 REPLIES
Five Stars

Re: How can I set mapreduce/hive job parameters in a tHiveInput query?

In the tHiveConnection component, click the 'Advanced settings', click the '+' under 'Hive Properties' and for each property:

Property

"hive.execution.engine"

 

Value

"tez"

Four Stars

Re: How can I set mapreduce/hive job parameters in a tHiveInput query?

Yes. That works. But we have parameterized the query and we want to parameterize those   "HIVE Set params" through a context variable. That way we can set different set of parameters for different hive queries. This way we have to hard code it for all the queries.