tFileInputDelimited with malformed data

One Star

tFileInputDelimited with malformed data



I have a 100000 line CSV file (tFileInputDelimited), which I'm trying to process with Tmap although experiencing an issue with one line:


"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"


It creates extra columns, so the data no longer matches the schema


We're using CSV options:
Field separator: ","
Escape Char: "\"" (have tried different options)
Text Enclosure: "\""


Cheers, M

Thirteen Stars

Re: tFileInputDelimited with malformed data

@mmeckens,you have extra Field separator,which were using as a Field separator.

You nee to correct the line in input csv file by manually.

below line data is not in proper csv format.,you need to correct it.

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"



Manohar B
Fifteen Stars TRF
Fifteen Stars

Re: tFileInputDelimited with malformed data

If this issue arrives "sometimes", you may prefer to correct it automatically instead of manually.
For that, use a tFileRow component to read the file line by line, then using a tMap or tJavaRow, you will be able to reformat the line (maybe using some regex) to generate a well formated new file.


Cloud Free Trial

Try Talend Cloud free for 30 days.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.