LOOP TFILE LIST UNTIL ALL FILES ARE READ

LOOP TFILE LIST UNTIL ALL FILES ARE READ

Hi
i sometimes have to bring in data from a load file, at the moment I use tfielist to check for the file in a directory,
if (Integer)globalMap.get("tFileList_1_NB_FILE") > 0 i proceed put these details in to a DB table.

However sometimes i get more than one of these load files , at the moment tfilellist will only pick up one (the first one) and populate the TABLE, and on my next run (cron) it will pick up the next .
I want to be able to loop my job until his until all files are populated to the respective table , and from there continue

regards

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

Hi wesleychristelis
Instead of writing the name of the file u can only write the extension of the file in tfilelist and process. plz check attached images .

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

Hi
unfortunately if I do the above mentioned i will pull in other txt files as there are other txt files that get picked up with other names ... so i have to use the criteria of

*CustomerLoad*.txt to differentiate the files
i Have a status in the file that I read , if it is a FULL (Complete) load (multiple files)i need to keep running te job until all files are read and inserted into a DB, otherwise it will be a DIFFERENTIAL load and i need on read that differential (a single file).
at present I start the load job and at the end I check what tye of load it was, if it is a FULL load I call a tRunJob which calls another job this other job in essence calls the load job again and it will loop until all files are read and prove false ,(tRunJob is not called again).
I personally think tis is a kludge way of doing it (but it works for now) is there no way to loop the job internally (ie not calling another job)
i trust this makes sense...
Regards

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

Hi wesleychristelis
Connect the tFileList_1 --iterate--> tFileProperties
This will loop through all the files found.
You do not need to add the condition :
if (Integer)globalMap.get("tFileList_1_NB_FILE") > 0
as if there is no files found it will not iterate through any files, the job will simply end.
Regards,
Brandon
Seven Stars

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

If you want to kill the tFileList loop after processing a file that meets your condition, then link tFileList using a second iterate connection to a tJava containing the following code: if (<your condition>) break;
So your job would be:
tFileList (loop through all files that you might want to process) --iterate1--> process each file found
--iterate2--> test your condition and kill the loop if appropriate
One Star

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

hye..my i know,why my output cannot display the data. i use this for tfileproperties ((String)globalMap.get("tFileList_1_CURRENT_FILE"))
connecting to socket on port 4056
connected
||||||
||||||
||||||
||||||
||||||
disconnected
One Star

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

File operation is very important in data integration. In this tutorial I am going to explain how to get list of files and folders name from a directory or a sub directory.
tFileList component in Talend Open studio is used for listing of all files and directories. You can iterate it and get list of all files, directory from current folder as well as sub directory.
http://talendtutorials.com/talend-interview-question/how-iterate-filename-and-directories-talend
One Star

Re: LOOP TFILE LIST UNTIL ALL FILES ARE READ

Hello,
i need some help, i have a folder content files csv , i want to read all this files csv by they name,
i want to divide them groups of 12 and then for each group i want to make the average some one have a jobs talend for that.
i used tFileList but i didnt know haw use it for my exemple
thanks,