Batch Data transfer from database

One Star

Batch Data transfer from database

Hi,
What is the best way to transfer data from database to hdfs using Talend? Is there a batch process that transfers the entire data to a single hdfs file, so I could run the job and close the talend application? Or Do I have to split the huge data into several transfers?
Thanks.
Highlighted
Four Stars

Re: Batch Data transfer from database

Hi sharidos - take a look at squoop. You can use the Talend Sqoop component to bulk transfer data from an rdbms to hdfs and vice versa
See https://help.talend.com/search/all?query=tSqoopImport&content-lang=en
One Star

Re: Batch Data transfer from database

Can Talend's sqoopimport component be used for Synonyms? It works fine with tables.
Is there any specific configuration required for Synonyms?
Thanks.
One Star

Re: Batch Data transfer from database

I am using TOS .5.4.1. Is it possible to send the output of tsqoopimport to thbaseoutput? Does it need any other component in between?
I want to transfer data in batches. Like first batch with x number of rows and the second batch with another x number of rows. Is there a way to automate it in single job submission?
Thanks.

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch