UTF-8 (csv) files that have issues with \n newline issues within the files (see attached) See Row 3 of the Preview where the \n is causing the file to go to new line. How to fix? Notepad++ works to find/replace /n to " " but i don't want to keep doing this every month for 50 files to upload. I tried tmap row3.COLUMN.replaceAll("\\r\\n|\\r|\\n"," ") and did this for every column, however, then I am getting a nullexception after it runs half of the file.
Update: I got the replace to work now and the data looks good, but i had to add a tFilterRow_1 before tMap because there are 2 rows at the bottom of every file in the first column (summary info of the report) and I filtered them out. Is there a better way of filtering out these 2 rows in every file, because it made the length of uploading 4xs longer????
Latest Update: Instead of using tfilter, i am using tMAP to filter out the 2 rows and it runs pretty fast. However, I am trying to iterate through every file and have each file automatically loaded into oracle, but it doesn't seem to be working. When i copy/paste the last 3 processes and create them individually they run perfectly (see individual processes in green in bottom right). The only different is the file path in the csv and one does a create table and the other is doing a insert into and i am not seeing anything wrong with the tMap. Any Suggestions?
If you always want to filter out the last two rows from an input file, you can set the "Footer" property on the file input to "2". This will always ignore the last two rows and you can get rid of the tMap (as far as I understood your initial problem). I would assume the tMap is somehow misconfigured but I cannot tell you how without seeing it.
Thanks JoRoesecke, I didn't think about that, that may help for one part of it. I am using tMAP because i need to remove /n newline characters from the csv file. My latest with this is that I have found that I can get the Job to run fine (see attached) when i have my File Delimited set to a specific file path on the share drive, but once i put ((String)globalMap.get("tFileList_1_CURRENT_PATH")) that is when the tMAP is crapping out on me. I tried to put footnote to 2 and I tried tJavaRow instead of tmap to replace my /n characters, but i am not getting NULL Pointer Exception.