I am creating big data spark job and want to load data into dynamic partitioned hive tables.
Which component I can use to load data into hive and what would be the workflow?
1.tHDFSConnection set your connection to your Hadoop cluster
2.set connection to hive in HIVE component
3.tHiveRow drop the table if present
4.tHiveRow create external table table
5. tFileInutDelimiter load the data to HDFS external table location
6. tHDFSoutput to load data to external table path
Which version you are using? With 6.2.1, all these tHive component is not available. Only tHiveConfiguration, tHiveInput and tHiveOutput is there.
Data is loaded using spark job but facing problem when the Hive table is dynamically partitioned.
I am using Talend open studio for Big data Version TOS_BD-20150508_1414-V5.6.2 . try to Download those components from https://exchange.talend.com/.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Take a look at this video about Talend Integration with Databricks
Learn how<SPAN>to modernize your Cloud Platform for Big Data Analytics with Talend and Microsoft Azure</SPAN>