Use Python Script in Hive Components

Seven Stars

Use Python Script in Hive Components

Hi everyone,

 

I'm using Big Data Sandbox 6.3_1.7 version. I was able to use hive components to achieve most of requirements (External tables, partition table, bucketing, etc). One last requirement I have is to use a python script as a part of my hive code. I tried executing add FILE /home/Desktop/my.py; command using tHiveRow component... but it did not really work. 

Can someone help with an example on this please?

Tags (1)
Moderator

Re: Use Python Script in Hive Components

Hello,

Sorry for delay. We have redirected your issue to our bigdata experts and then come back to you as soon as we can.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Use Python Script in Hive Components

Hi,

 

Thank you very much for looking into this query and raising it with your team. I'll wait for the response. 

Seven Stars

Re: Use Python Script in Hive Components

Hi,

 

Any updates on this query? Is there anyway this can be a feature request? 

One Star

Re: Use Python Script in Hive Components

Python can be used as a UDF from Hive through the HiveQL TRANSFORM statement. For example, the following HiveQL invokes the hiveudf.py file stored in the default Azure Storage account for the cluster.

add file wasb:///hiveudf.py;

SELECT TRANSFORM (clientid, devicemake, devicemodel)
    USING 'D:\Python27\python.exe hiveudf.py' AS
    (clientid string, phoneLable string, phoneHash string)
FROM hivesampletable
ORDER BY clientid LIMIT 50;
Seven Stars

Re: Use Python Script in Hive Components

Interesting, thanks for sharing.