Six Stars

tFileInputDelimited not read all fields of a data row

Hi all,

I have a data row in CSV file, I create a simple job to print it onto the console, but the tFileDelimited does not read all fields of the row.

The last field of the row is a Date value but when print it on console, the last value has been printed be "0.99"., I don't know what field it belong to.

How to read all the fields ?

I have tFileDelimited like this:

Capture.PNG

I want to keep the data format so CSV option was not checked and nothing checked in advanced settings

Below is the data row.
"9806699","DATATESTLUVINA8","2017-01-01 08:00:00","0990","0990","09.9","09.9","09.9","09.9","0990","09.9","09.9","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","09.9","0990","0990","0990","0990","0990","0990","0990","09.9","0990","""Lg,Mx""16""","0990","100001","100002","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","0.99","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","0990","0990","0990","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","1000016666444488.899","1000016666444488.899","0990","0990","0990","0990","0990","0990","09.9","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","09.9","0990","","100003","0990","0990","0990","100005","100004","100005","0990","09.9","0990","09.9","0.99","0.99","09.9","09.9","09.9","09.9","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0.99","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","09.9","0990","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","0990","0990","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0990","0.99","0990","0.99","0990","0.99","0990","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0.99","0.99","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0.99","0.99","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","0.99","0990","0.99","0990","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0.99","0990","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0.99","09.9","100006","1000016666444488.899","0990","1000016666444488.899","09.9","0990","0990","""Lg,Mx""16""","0990","0990","1000016666444488.899","1000016666444488.899","0990","1000016666444488.899","1000016666444488.899","09.9","09.9","100007","0.99","0990","0990","0.99","""Lg,Mx""16""","0990","0.99","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","0990","0990","0990","0990","09.9","0990","0.99","0990","0990","09.9","0990","0.99","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","0.99","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","09.9","09.9","09.9","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","0.99","099","099","099","099","09.9","09.9","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0.99","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","09.9","09.9","09.9","09.9","09.9","0.99","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","100008","100009","100010","","2017-01-03 06:00:00.000"

 

 

 

 

  • Data Integration
6 REPLIES
Eleven Stars

Re: tFileInputDelimited not read all fields of a data row

A couple of things to check....

1) Have you set up enough columns in your metadata? I estimate (from a quick comma per row count) that you have over 1000 columns. Have you tried to add some more to see what result you get?

2) Do you have any carriage returns and line feeds in the data that you are not aware of? Your component config uses \r\n to separate rows. If it finds these, it will just start the next row. Since you are not using the CSV options, if you have \r\n in quoted text, this will be considered a new row.

Rilhia Solutions
Six Stars

Re: tFileInputDelimited not read all fields of a data row

Thank you,
1, I'm sure that the schema is enough column because other data row is print OK.
2, My data not have \r\n in quoted text, as you can see at the sample data I post.
I think my data row is out of max length which tFileInputDelimited can read by row.
Does Talend set the max length of each data row that tFileInputDelimited can read ?
Nine Stars TRF
Nine Stars

Re: tFileInputDelimited not read all fields of a data row

Check your schema definition


TRF
Six Stars

Re: tFileInputDelimited not read all fields of a data row

I'm sure that the schema is enough columns because other data rows is print OK.
Nine Stars TRF
Nine Stars

Re: tFileInputDelimited not read all fields of a data row

Do you have values like this one in other records?

"""Lg,Mx""16"""

TRF
Eleven Stars

Re: tFileInputDelimited not read all fields of a data row

I don't think I was very clear, the \r\n I was talking about are invisible. That text is the Java representative of a carriage return and line feed or simply hitting the "ENTER" key

like that.

 

@TRF raises a good point about the values you are expecting. If you are getting some rows returned with the correct number of columns and some which do not return the correct number of columns (or expected last column), it will likely mean one of the following things....

 

1) You have a carriage return and line feed in your row (some free text with the "ENTER" key having been pressed)

2) You have an extra comma in your free text (most likely)

 

There are online tools that will count the number of commas are in your rows. Try testing a "good" row against a "bad" row 

Rilhia Solutions