[resolved] Skip duplicate file to process

One Star

[resolved] Skip duplicate file to process

Hi
I have one talend job.
First, sub job process all the files in tfilelist folder and reject all files having not required records or format. And convert that file into .csv.bad extension.
After this main job starts working, and process all the files not converted to .csv.bad files. And convert those files into .csv.read.
My requirement is, if tfilelist receive and file which is already processed and converted to .csv.read, skip that file to process.
Any help ?

Accepted Solutions
Six Stars

Re: [resolved] Skip duplicate file to process

To do that, after your tFileList, iterate to a tFileExist with current filename suffixed to ".read". 
If exist rejected else continue ...

All Replies
Six Stars

Re: [resolved] Skip duplicate file to process

Hi,
Can you display screens from your job please ?
Thanks
One Star

Re: [resolved] Skip duplicate file to process

Hi arnaudbroens
Here is my job
prntscr(dot)com/969t12
One Star

Re: [resolved] Skip duplicate file to process

Did you set the file extension in the tFileList into "*.csv"?
Six Stars

Re: [resolved] Skip duplicate file to process

You have many options on tFileList. Check your options and specially extensions ( says junmilsso ).
For extensions, if you select 'case sensitive : true', write "*.csv" and "*.CSV"
One Star

Re: [resolved] Skip duplicate file to process

Yes I did
Files comes up with dates
For Example: XYZ_25122015.csv
If file is ok then that would be converted into XYZ_25122015.csv.read
But if file is not ok then that would be converted into XYZ_25122015.csv.bad without entering any single record into database.
My question is, if any file come up with same date which already has been  processed and converted into  XYZ_25122015.csv.read should be rejected.
In both tfilelist, I have set the mask like this "*.csv"
Any suggestion, how it can be done ?
Six Stars

Re: [resolved] Skip duplicate file to process

To do that, after your tFileList, iterate to a tFileExist with current filename suffixed to ".read". 
If exist rejected else continue ...
One Star

Re: [resolved] Skip duplicate file to process

Hey arnaudbroens
Job is done now, thanks for reminding me tfileexist component.
Its been long time that I didn't use that component. Smiley Happy
Best !!