One Star

Transfer data between Hive and SQL Server

Hi,
If I have this architecture :
- VM1 : VM for Talend server (jobs execution)
- VM2 : VM for hadoop cluster (with Hive database)
- VM3 : VM for SQL Server database.
If i want to make a Talend job that transfer data from Hive to SQL Server. The job is executed by the VM1.
Imagine my job is simply :
tHiveInput -------> tMSSqlOutput
The transfered data are directly transfered between VM2 (Hive) and VM3 (SQL Server) ?
or
The transfered data are transfered between VM2 (Hive) and VM1 (Talend). Then, transfered between VM1 (Talend) and VM3 (SQLServer) ?
Thanks in advance.
Tangi
1 REPLY
Seventeen Stars

Re: Transfer data between Hive and SQL Server

If you use this architecture the data are transferred over the VM1. What else would you expect?
You could also use scoop to transfer the data directly from the hive table to the database - without the talend job or you can build the Talend job and transfer it to the VM2.
I suggest the scoop approach.