One Star

Taneld Spark job ThiveInput and ThiveOutput not working on Million plus rows

Version: Talend Real-time Big Data Platform 6.2.1

So im trying to pull data from one hive table to another using thiveinput(query) to thiveoutput(create a table) with a Spark job. If its any table under like a 1000 rows it works fine (a 1000 rows is just what I have noticed it might be higher or lower). If I try and run a query on a table over 1000 plus rows it looks like it runs the query but no data is ever put into the output table. If I run the query in hive I get all the data back just cant seam to get the spark job to read and/or write the data to the output table/file. The only difference is on the larger tables I get the below message in the talend log. I am new to Talend and Spark and not sure what I could be doing wrong
Thanks

GetLocalGroupsForUser error (1332): No mapping between account names and security IDs was done.

  • Big Data
1 REPLY
Moderator

Re: Taneld Spark job ThiveInput and ThiveOutput not working on Million plus rows

Hello,

With you subscription solution Talend Real-time Big Data Platform 6.2.1, could you please create a case on talend support portal? In this way, we can give you a remote assistance(webex) through support cycle with priority to see if it is a performance issue on Talend Real-time Big Data Platform 6.2.1.

https://login.talend.com/support-login.php.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.