I am new to talend. I am trying to build a job which will connect to mapr stream , consume the data and write the data to a hive table.
I am using maprstream input to consume the data which is consuming the data correctly. but when inserting the data into hive using thiveoutput component its not working. I am even not getting any error. Can anybody help here.
Are you Posting to partitioned Hive table (in Append Mode) ? can you please attach the screen shots of the Job and the configurations of the tHive output ?
No I have not selected partitioned option. And save mode is Append. I am using 6.4 Real time big data platform.
The job is a big data streaming job, with one component tMapRStreamInput Input and the other one is tHiveOutput.
I can see the messages are getting consumed using the tMapRStreamInput, but I can't insert them to hive tables.
Note: Cant attach the screenshot because of organisation security policy.
were the Hive tables created upfront before appending the data ? I have this issue when the Hive tables weren't created upfront and tried to append the data to it.
can you able to print the data to the console ? if so, i don't see the reason why it won't get inserted into a Hive table.
By the explanation the only thing that can go wrong is the Hive table schema created upfront should match the parsing schema.
Yes, I am able to print the data in console. The only thing I am wondering i there are any schema mismatch error, then it should show in the console.
schema mismatch will print the data to the console and it doesn't post the data to the hive table, make sure the schema's are defined and i would prefer in using lower case letter's. Please post if that solved the issue after trying that
do you mean the tHiveOutput component will itself print the data in case its not able to insert the data. Then its not printing form me as well. I was different component to print the data.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to make your data more available, reduce costs and cut your build time
Read about OTTO's experiences with Big Data and Personalized Experiences
Take a look at this video about Talend Integration with Databricks