One Star

[resolved] How can I export tables from MySQL to Hive/Hadoop?

How can I export tables from MySQL to Hive/Hadoop? Which components will help? like tHiveConnection,tHiveRow or tHiveClose?
Thank you for your replies.
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Please note that a tHiveInput component is planned to get released on 5.0.
5 REPLIES
Employee

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Hi,
Absolutely, you can combine a tMySQLInput and a tHiveRow to do that!
It would look like that:
tHiveConnection
|
|
v
tMySQLInput ----> tHiveRow
|
|
v
tHiveClose
One Star

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Could you provide more detail on how to use the tHiveRow component here? You can't do a single-row insert in Hive/HQL.
Thanks.
---Paul
One Star

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Could you provide more detail on how to use the tHiveRow component here? You can't do a single-row insert in Hive/HQL.
Thanks.
---Paul

vgalopin is right,but this method is inefficient,data is imported one by one.Actually, I recommend "load overwrite" command used in tHiveRow's query like this "load data local inpath '"+context.hive_file_dir+"t_test.txt' overwrite into table t_test"
One Star

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Through using this tool(Talend),I find it supports hive/hadoop not very well.Free 4.2.3 version only provide three components,which are tHiveConnection,tHiveRow and tHiveClose.
Employee

Re: [resolved] How can I export tables from MySQL to Hive/Hadoop?

Please note that a tHiveInput component is planned to get released on 5.0.