Five Stars

How can I use the tIterateToFlow

hello, 

I have to get  the files (source_file.csv) and (map_file.xml) from the local directory,  put them on the server and after that in a database. 

so I have create the ftp connection and the tFileList, tFTPPut and it works perfectly.

My problem is  the IterateToFlow, I wanna get two flux, the first one with the XML files and the seconde wit the csv files. how can I do it please ?

 

Thank you 

Fath

5 REPLIES
Fifteen Stars

Re: How can I use the tIterateToFlow

You have to read in the data using a tFileInput... component. The tFileList simply supplies the filename or the filepath. It does not provide the file content.

Rilhia Solutions
Five Stars

Re: How can I use the tIterateToFlow

Hello rhall_2_0, 

Thank you for your answer

 

I'm trying to get the file name too on two columns . 

column 1 named "map_file" will contain: the file name ending by ".XML"

column 2 named "source_file" will contain: the file name ending by ".csv"

is that possible with the tIterateToFlow component please? 

Fifteen Stars

Re: How can I use the tIterateToFlow

You are missing out a lot of steps. First of all you have to retrieve your files. Take a look here (https://help.talend.com/reader/jomWd_GKqAmTZviwG_oxHQ/MhJ3ofNo_IPaoAWAAsxYEQ) for a guide on how to use the FTP components. 

 

Next you need to read the data into the file you have retrieved. Take a look here (https://help.talend.com/reader/KxVIhxtXBBFymmkkWJ~O4Q/Kbz5WfMuxJ_iK6mJi4Fivg) for a guide on how to read CSV files. Take a look here (https://help.talend.com/reader/QgrwjIQJDI2TJ1pa2caRQA/r~_1QgkIuMA7jQsYkIUnZg) for a guide on how to read XML documents.

Rilhia Solutions
Five Stars

Re: How can I use the tIterateToFlow

Thank you again!

I probably did not speak well; I am not trying to read the contents of the csv and xml files, I am trying to parameterize the names of the files in a context variable, which I would later give to another tRun job that will run in a loop until all files are processed.

I want to have both boxes in  set instead of typing the name of the map_file and the source_file each time (the screenshot attached)

Fifteen Stars

Re: How can I use the tIterateToFlow

Ah I see. OK, the tFTPFileList will return the filenames one at a time. So you will get one file per iteration. If you want to supply your files in pairs (one csv and xml) to another Job at the same time, you will need to load all of the files found to a tHashOutput component and then (in another subjob) read them in to a tMap twice (one tHashInput for the main and one for a Lookup). This will require you to work out some natural way of linking the pairs (maybe filename??). You will also need to filter your Main (lets say the main is used to bring back CSV files) and your Lookup (for XML files). This will mean you get your CSV file list driving the tMap. The Lookup will supply matching XML files and you can output the CSV and XML filenames in pairs per row out of the tMap.

The key things to work on will be a natural way of linking the files, and filtering of the data returned by the tHashInput components.

Rilhia Solutions