tFileInputDelimited reads fewer rows than lines in the file

Highlighted
Four Stars

tFileInputDelimited reads fewer rows than lines in the file

I have a UTF-8 csv file with 248,995 LF-separated lines (header included).

tFileInputDelimited reads only 248,980 data rows, no rejects.

What should I be looking for in the data? Or in the component?

Highlighted
Sixteen Stars
Sixteen Stars

Re: tFileInputDelimited reads fewer rows than lines in the file

Maybe some lines with 1 or more LF in the middle.

TRF
Highlighted
Four Stars

Re: tFileInputDelimited reads fewer rows than lines in the file

Thank you, TRF.

Excel and Talend (if CSV specified) are able to keep together in one field anything between two "s, including control characters.

So even if the number of lines of a .csv file (as read in notepad++) is higher than the number of rows read with Excel are different (adjusted for headers), the data rows are correct.

Not sure what the end system does with the LF's in a field...

Highlighted
Sixteen Stars
Sixteen Stars

Re: tFileInputDelimited reads fewer rows than lines in the file

If your fields are contained between "s and you use Notepad++ to count lines, it's normal to have a difference with tFileInputDelimited as this one counts logical records, not physical lines.
If you count lines with tFileRowCount or read files with tFileInputFullRow you will have the same count as with Notepad++.

TRF

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Talend Cloud Developer Series – Defining Metadata

This video focuses on different methods of adding metadata to a job in Talend Cloud

Watch Now

Talend Cloud Developer Series – Updating Context Variables

This video will show you how to add context parameters to a job in Talend Cloud

Watch Now