Extract a billion records using Talend Data Integration

One Star

Extract a billion records using Talend Data Integration

Hi,
May i know is this TOS for data integration might have a capability to extract a billions records?
I need to extract out a billions of records from Oracle/db2 database.
Anyone have any experience on this? Smiley Sad
Regards,
Ommie
Moderator

Re: Extract a billion records using Talend Data Integration

Hi,
Here is a similar topic Forum 32192.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seventeen Stars

Re: Extract a billion records using Talend Data Integration

Talend does not have a limitation in processing number of row. You can process as many rows as you want/have. It is always a matter of efficiency and resources. Billions of row - why not.
I would always try to separate a chunk of rows and avoid trying to process all rows at once.
Processing billions of rows without separating it in chunks will lead into a nightmare on the productive system - regardless which tool do you use to do that.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch