I'm trying to process EDI records using Talend Data Mapper. Their are thousands of records stored in a flat file, with one row per record, delimited by \n. I'm passing the whole file as input to the mapper, and mapping the records to a delimited file. I'm getting this error:
End of file was found in the middle of an EDI segment/display
There are thousands of these errors, and the result is that no records actually get mapped. I'm not sure what to do. Can you send a file with multiple records to the mapper, or do you need to split it into one file per record first (and how would you do that)?
Unfortunately, no, as this is health data. I think I know what is happening, though: I've been sending all the EDI records through the mapper in one large file, and I think when it encounters a bad record, it logs the error, but then keeps reading from that spot. This would explain the cascading failures, since all the records after the bad one would be out of sequence.
When I split the data file so that the mapper only reads one record at a time, I was able to map about 75%, which is consistent with what I know about this data. I did that with tFileOutputDelimited, which lets you specify how many lines you want in each output file. There's probably a more elegant way, but this is working for me for now.
What kind of Input and Output Components you are using in Talend DI job???
you are directly performing Test run on the Out put structure???
if you running from DI job you have to use "tFileInputRaw---->tHMap--->tFileOutputRaw".
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema