One Star

Iterating through files

I have a job right now that loads three CSV files into a DB and then run a series of jobs against that to build out a data warehouse. In the future there may be as many as 5 or 10.
Right now I have a separate job running for each CSV file. What I would like is a single job that goes through a folder running on each CSV that it finds. Does someone have a sample on this?
Thanks,
Luke
4 REPLIES
One Star

Re: Iterating through files

The component you want is tFileList. In that component you can specify a file mask and hook the iterate output to the rest of your processing.
One Star

Re: Iterating through files

Hi, does anyone know if there is a way I can pass the output from tFileList into a tRunJob component?
One Star

Re: Iterating through files

Ah, looks like you can add a context variable in the job, then set it as a parameter to the tRunJob and then pass this through an iteration between the tFileList and tRunJob:
((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))
One Star

Re: Iterating through files

I've set up a cron job that pulls AppStore Sales reports to a folder,
which I then let a reoccurring Talend job:
looking in, pulling out files, extracting, loading into a staging table, moves the used files.

-- Update --
Hmm, can't see the files I uploaded...