Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Max limit size for Delimited file

Highlighted
One Star

Max limit size for Delimited file

Hi,
What is the maximum limit of number of records can be inserted into a .csv file using tFileOutputDelimited component.. ?
I am using Talend Enterprise Big Data 5.5.1 edition.
Highlighted
One Star

Re: Max limit size for Delimited file

Errr, there is a limit? I didn't know there was one. I've done 74 million recently, that's my largest I think - Not sure if that helps answer your question but if there is an upper limit I would also like to know that.
Highlighted
One Star

Re: Max limit size for Delimited file

Thanks Pem. I am just trying to insert 9 million records. I just wanted to know whether there is a limit.
Does anyone know whether there is a limit.
Highlighted

Re: Max limit size for Delimited file

at talend com the only limit is yourself!
There is no built-in limit to the number of records you can output with a tFileOutputDelimited. Once you get into really HUGE (1B+) numbers of records, you do need to start considering resources on the server and may find that your particular job and hardware configuration imposes a limit on the number of records you can process ( disk space, memory, etc... )

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog