Line separator in the middle of a line

One Star

Line separator in the middle of a line

Hello,
I have a problem with an input text file. I want to import the datas in a DB Table and I configure tFileInputDelimited with "\n" for the line separator and ";" for the filed separator.
But for some lines in my file, there is a text field surrounded with doubles quotes and there is a new ligne in the field so it break the structure.
When, in the tFileInputDelimited in the advanced configuration, I checked "Check the structure of all the lines according to the schema" (not sure of the translation, I use the french version) I saw the "brokens" lines (which appears in the "reject" of the tFileInputDelimited.
But I absolutely don't know how to handle it to have the brokens lines corrected (in fact to correct them I should delete the unwanted new line character but I don't know how to do that).
Thank you for your help

Re: Line separator in the middle of a line

Can you post sample contents of the file to help us understand more about your query?
Seventeen Stars

Re: Line separator in the middle of a line

The only way to solve that is to enclosure the fields with chars like ".
The tFileInputDelimited component can handle this and returns the line break within the enclosure as content.
"field1";"field2";"field3"
"field12";"field with
line break";"field31"
with returned as
field1|field2|field3
field12|field with\nline break|field31
Without enclosure you have no chance to parse that correctly.
Seven Stars

Re: Line separator in the middle of a line

Use "CSV Options" in tFileInputDelimited to allow line-breaks and ; within your fields to be handled (provided the field is enclosed as you say). You can then use tReplace to remove the unwanted line-breaks from your fields.
One Star

Re: Line separator in the middle of a line

Thank you for your responses.
In fact I used the "delimited" type so I don't have access to the CSV options but it works perfectly well when I changed my metadata to "CSV"