One Star

tFileInputDelimited with malformed data



I have a 100000 line CSV file (tFileInputDelimited), which I'm trying to process with Tmap although experiencing an issue with one line:


"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"


It creates extra columns, so the data no longer matches the schema


We're using CSV options:
Field separator: ","
Escape Char: "\"" (have tried different options)
Text Enclosure: "\""


Cheers, M

Twelve Stars

Re: tFileInputDelimited with malformed data

@mmeckens,you have extra Field separator,which were using as a Field separator.

You nee to correct the line in input csv file by manually.

below line data is not in proper csv format.,you need to correct it.

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"



Manohar B
Forteen Stars TRF
Forteen Stars

Re: tFileInputDelimited with malformed data

If this issue arrives "sometimes", you may prefer to correct it automatically instead of manually.
For that, use a tFileRow component to read the file line by line, then using a tMap or tJavaRow, you will be able to reformat the line (maybe using some regex) to generate a well formated new file.