Hi all,
I have a data row in CSV file, I create a simple job to print it onto the console, but the tFileDelimited does not read all fields of the row.
The last field of the row is a Date value but when print it on console, the last value has been printed be "0.99"., I don't know what field it belong to.
How to read all the fields ?
I have tFileDelimited like this:
I want to keep the data format so CSV option was not checked and nothing checked in advanced settings
Below is the data row.
"9806699","DATATESTLUVINA8","2017-01-01 08:00:00","0990","0990","09.9","09.9","09.9","09.9","0990","09.9","09.9","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","09.9","0990","0990","0990","0990","0990","0990","0990","09.9","0990","""Lg,Mx""16""","0990","100001","100002","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","0.99","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0990","0990","0990","0990","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","1000016666444488.899","1000016666444488.899","0990","0990","0990","0990","0990","0990","09.9","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","09.9","0990","","100003","0990","0990","0990","100005","100004","100005","0990","09.9","0990","09.9","0.99","0.99","09.9","09.9","09.9","09.9","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0.99","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","09.9","0990","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","0990","0990","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0.99","0.99","0990","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0.99","0.99","0.99","0990","0990","0990","0.99","0990","0.99","0990","0.99","0990","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0990","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0.99","0.99","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0.99","0.99","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","0.99","0990","0.99","0990","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","0.99","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0.99","0990","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0.99","09.9","100006","1000016666444488.899","0990","1000016666444488.899","09.9","0990","0990","""Lg,Mx""16""","0990","0990","1000016666444488.899","1000016666444488.899","0990","1000016666444488.899","1000016666444488.899","09.9","09.9","100007","0.99","0990","0990","0.99","""Lg,Mx""16""","0990","0.99","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","""Lg,Mx""16""","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","09.9","09.9","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","""Lg,Mx""16""","""Lg,Mx""16""","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","09.9","0990","0990","0990","0990","09.9","0990","0.99","0990","0990","09.9","0990","0.99","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","0.99","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","09.9","0990","09.9","0990","09.9","09.9","09.9","0990","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","09.9","09.9","09.9","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","""Lg,Mx""16""","0.99","0.99","099","099","099","099","09.9","09.9","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","""Lg,Mx""16""","""Lg,Mx""16""","0.99","0.99","09.9","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","09.9","09.9","09.9","09.9","09.9","0.99","0990","0990","0990","0990","0990","0990","0990","0990","0990","0990","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","0.99","100008","100009","100010","","2017-01-03 06:00:00.000"
A couple of things to check....
1) Have you set up enough columns in your metadata? I estimate (from a quick comma per row count) that you have over 1000 columns. Have you tried to add some more to see what result you get?
2) Do you have any carriage returns and line feeds in the data that you are not aware of? Your component config uses \r\n to separate rows. If it finds these, it will just start the next row. Since you are not using the CSV options, if you have \r\n in quoted text, this will be considered a new row.
Check your schema definition
Do you have values like this one in other records?
"""Lg,Mx""16"""
I don't think I was very clear, the \r\n I was talking about are invisible. That text is the Java representative of a carriage return and line feed or simply hitting the "ENTER" key
like that.
@TRF raises a good point about the values you are expecting. If you are getting some rows returned with the correct number of columns and some which do not return the correct number of columns (or expected last column), it will likely mean one of the following things....
1) You have a carriage return and line feed in your row (some free text with the "ENTER" key having been pressed)
2) You have an extra comma in your free text (most likely)
There are online tools that will count the number of commas are in your rows. Try testing a "good" row against a "bad" row