tFileOutputDelimited writing more files than it should
Hi, I was experimenting with creating a job in talend and ran into a problem where tFileOutputDelimited is writing extra empty files, even when the option to not create empty files was selected. My job's flow is like this: I iterate through all of the .zip files in a given folder and unzip them in a temporary folder with the same name as the .zip file (for example, all of the files in the zipped.zip file would be unzipped to the \tmp\zipped.zip folder). I then iterate through all of the unziped files in that folder and check if that particular file has already been processed (I use a tOracleConnection to connect to a database and a tJava to run the query, the return is saved in a context variable). Afterwards, in case the file is not a duplicate, I get the whole content of each file with a tFileInputRaw component as I want to be able to process its contents as a whole. Then in a tJavaRow component I process the flat file content and turn it into a String which then is sent to the output as an Object. After that I split the flow into 3, one of the flows would simply use a tFileOutputDelimited to write the original file somewhere else, the other two would process the String further. That was my original plan, but I got stuck in a place I didn't expect. When testing the flow that convertsthe String back to a String and outputs it using the tFileOutputDelimited component, I noticed that it was creating more files than it should. In the .zip file that I use for testing I have two .lis flat files. One of them is a duplicate, the other is not. So that the flow would not process duplicate files, I have a tJavaRow component that, before processing the file, checks if they are duplicate, if they are then it uses continue to skip that iteration. But the tFileOutputDelimited created both the duplicate and the non-duplicate files, even though the duplicate file should not have gotten that far into the flow. The duplicate file gets created empty, so I checked the 'do not create empty files' option in the component, but it did nothing as far as I can tell. Then I though that maybe the continue didn't help and the duplicate file managed to get to the end of the flow but empty, I did some tests just before the tFileOutputDelimited component and found out that no, it never got to that point, that part of the flow only ran once and when it ran was in the iteration where the tFileList component was iterating the non-duplicate file. This is my job:
If anyone has got any idea as to why this is happening, I would appreciate it. Best Regards, Tiago Vieira
Re: tFileOutputDelimited writing more files than it should
Adding to this, I now tried using continue in all cases, regardless if the file is duplicate or not, and I found that the flow always creates the first file it processes. Edit: Actually, it creates all of the files, the only difference is that the duplicates are created empty.