Batch Data transfer from database

One Star

Batch Data transfer from database

Hi,
What is the best way to transfer data from database to hdfs using Talend? Is there a batch process that transfers the entire data to a single hdfs file, so I could run the job and close the talend application? Or Do I have to split the huge data into several transfers?
Thanks.
Highlighted
Four Stars

Re: Batch Data transfer from database

Hi sharidos - take a look at squoop. You can use the Talend Sqoop component to bulk transfer data from an rdbms to hdfs and vice versa
See https://help.talend.com/search/all?query=tSqoopImport&content-lang=en
One Star

Re: Batch Data transfer from database

Can Talend's sqoopimport component be used for Synonyms? It works fine with tables.
Is there any specific configuration required for Synonyms?
Thanks.
One Star

Re: Batch Data transfer from database

I am using TOS .5.4.1. Is it possible to send the output of tsqoopimport to thbaseoutput? Does it need any other component in between?
I want to transfer data in batches. Like first batch with x number of rows and the second batch with another x number of rows. Is there a way to automate it in single job submission?
Thanks.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now