Batch Data transfer from database

One Star

Batch Data transfer from database

Hi,
What is the best way to transfer data from database to hdfs using Talend? Is there a batch process that transfers the entire data to a single hdfs file, so I could run the job and close the talend application? Or Do I have to split the huge data into several transfers?
Thanks.
Four Stars

Re: Batch Data transfer from database

Hi sharidos - take a look at squoop. You can use the Talend Sqoop component to bulk transfer data from an rdbms to hdfs and vice versa
See https://help.talend.com/search/all?query=tSqoopImport&content-lang=en
One Star

Re: Batch Data transfer from database

Can Talend's sqoopimport component be used for Synonyms? It works fine with tables.
Is there any specific configuration required for Synonyms?
Thanks.
One Star

Re: Batch Data transfer from database

I am using TOS .5.4.1. Is it possible to send the output of tsqoopimport to thbaseoutput? Does it need any other component in between?
I want to transfer data in batches. Like first batch with x number of rows and the second batch with another x number of rows. Is there a way to automate it in single job submission?
Thanks.