Talend Big Data: HIVE Table tHiveLoad OVERWRITE doesn´t work

One Star

Talend Big Data: HIVE Table tHiveLoad OVERWRITE doesn´t work

Hi guys,
I need to run a job in which I have to truncate every time an Hive Table and insert new data.
If I execute the query from the HUE browser interface using the same user like this:
INSERT OVERWRITE TABLE work.bi_xxx
SELECT ...
the map reduce job works properly. :-)
If I translate this into a Talend Job using a tHiveLoad and using:
- the SAME user
- and Load action: INSERT
- Action on file: OVERWRITE
it doesn´t work and I receive this exception: :-(
Default;6;Java Exception;tHiveLoad_2;java.sql.SQLException:Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask;1

If I execute the same but with Action on File: APPEND it works properly. :-)
How can be considering that I´m using the same user with the same permissions?
I tried also to use a tHiveRow component before in order to truncate the table and then using APPEND in the tHiveLoad but the TRUNCATE command doesn´t work.
Please can you help me?
I really don´t know why this different behavior.
Thanks in advance
Mary
One Star

Re: Talend Big Data: HIVE Table tHiveLoad OVERWRITE doesn´t work

Mary - does the APPEND work in HUE browser or in Talend using the Hive component?
One Star

Re: Talend Big Data: HIVE Table tHiveLoad OVERWRITE doesn´t work

HI,
yes, the APPEND in Talend works properly but not the OVERWRITE.
Do you have any idea?
Thanks in advance
Mary
One Star

Re: Talend Big Data: HIVE Table tHiveLoad OVERWRITE doesn´t work

Does the underlying HDFS files (or partitions) have necessary permissions for the Talend user.  Try giving 777 to the underlying HDFS file.  Also grant all on the table to the user running the job.
I am using 5.4 big data platform and seems to run fine.  The Hadoop distribution is Horton works HDP V1.2.
The SQL from Talend is:
INSERT OVERWRITE TABLE EDW_SHRD_VIEWS.DIM_DELTA SELECT NW.nm , NW.attr , NW.bgn_dt , NW.id , NW.end_dt , CASE WHEN OD.ID IS NULL THEN 'I' WHEN OD.ID IS NOT NULL AND OD.NM != NW.NM THEN 'UI' WHEN OD.ID is NOT NULL AND OD.ATTR !=NW.ATTR THEN 'U' ELSE 'N' END AS CHG_IND, OD.nm , OD.attr , OD.bgn_dt , OD.id , OD.end_dt  FROM  EDW_SHRD_VIEWS.STG_DIM NW LEFT OUTER JOIN  EDW_SHRD_VIEWS.DIM OD ON(  OD.id = NW.id  )