tFileInputDelimited with malformed data

One Star

tFileInputDelimited with malformed data


Hi,

 

I have a 100000 line CSV file (tFileInputDelimited), which I'm trying to process with Tmap although experiencing an issue with one line:

 

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

 

It creates extra columns, so the data no longer matches the schema

 

We're using CSV options:
Field separator: ","
Escape Char: "\"" (have tried different options)
Text Enclosure: "\""

 

Cheers, M

Forteen Stars

Re: tFileInputDelimited with malformed data

@mmeckens,you have extra Field separator,which were using as a Field separator.

You nee to correct the line in input csv file by manually.

below line data is not in proper csv format.,you need to correct it.

"Everest","Norgay, Tenzing "Sherpa";Hillary, Ed","Nepal","Khumbu"

 

 

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Fifteen Stars TRF
Fifteen Stars

Re: tFileInputDelimited with malformed data

If this issue arrives "sometimes", you may prefer to correct it automatically instead of manually.
For that, use a tFileRow component to read the file line by line, then using a tMap or tJavaRow, you will be able to reformat the line (maybe using some regex) to generate a well formated new file.

TRF

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now