[resolved] How to write every row from tParseRecordSet to a different delim file?

One Star

[resolved] How to write every row from tParseRecordSet to a different delim file?

Hello.
I have a requirement, where a stored procedure's resulting records must be saved in different files. In other words, if the SP returns 10 rows, each of them should be saved in different csv files, a total of 10 files. The files should be named after a particular column in the result.
I currently I have my setup like this:
tRowGenerator_1 --> tMSSqlSP_1 --> tParseRecordSet_1 --> tFileOutputDelimited_1  --> tLogRow_1

This works if I want all the data in a single file.
But I want them in different files, plus the files should be named based on a column, for example OrderID, in the SP results. How do I do this?
Many thanks for any one that can help.

Accepted Solutions
Highlighted
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

I donT have MSsql, but I was able to solve for:
"records must be saved in different files. ..., each of them should be saved in different csv files, ...
The files should be named after a particular column in the result. "
tRowGenerator -row1-> tFlowToIterate -iterate-> tFixedFlowInput -row3-> tFileOutputDelimited

tRowGenerator creates rows, each with a string and a number column: FileNameCol and NumberCol.
tFixedFlowInput uses a Single Table:
Column           Value
FileNameCol row1.FileNameCol
NumberCol row1.NumberCol

If you have a lot of columns, this tFixedFlowInput component might be a bit of a pain to set up. :-(
The File Name in tFileOutputDelimited is from row1, not row3!
"C:/Talend/TOS_DI-Win32-r92826-V5.2.0/workspace/out_" + row1.FileNameCol + ".csv"
(When I had used row3.FileNameCol, the file name and content were different (shifted one value) and the last file had the name out_null.csv.)

I would think that the following should work for your requirement:
tRowGenerator_1 --> tMSSqlSP_1 --> tParseRecordSet_1 --> tFlowToIterate -iterate-> tFixedFlowInput --> tFileOutputDelimited_1  --> tLog

All Replies
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

Please someone help. Thank you so much.
Five Stars

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

just use flow to iterate component to get each row from main flow and add that row in single file, used any column from tFlowtoIterate to named the file.
tRowGenerator_1 --> tMSSqlSP_1 --> tParseRecordSet_1 -->tFlowtoIterate---> tFileOutputDelimited_1  --> tLogRow_1

hope this will work for your requirement.
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

Thank you umeshrakhe.
I tried your suggestion, but it is not possible to connect tFlowToIterate with tFileOutputDelimited.
What should I do to link tFlowToIterate with tFileOutputDelimited?
Highlighted
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

I donT have MSsql, but I was able to solve for:
"records must be saved in different files. ..., each of them should be saved in different csv files, ...
The files should be named after a particular column in the result. "
tRowGenerator -row1-> tFlowToIterate -iterate-> tFixedFlowInput -row3-> tFileOutputDelimited

tRowGenerator creates rows, each with a string and a number column: FileNameCol and NumberCol.
tFixedFlowInput uses a Single Table:
Column           Value
FileNameCol row1.FileNameCol
NumberCol row1.NumberCol

If you have a lot of columns, this tFixedFlowInput component might be a bit of a pain to set up. :-(
The File Name in tFileOutputDelimited is from row1, not row3!
"C:/Talend/TOS_DI-Win32-r92826-V5.2.0/workspace/out_" + row1.FileNameCol + ".csv"
(When I had used row3.FileNameCol, the file name and content were different (shifted one value) and the last file had the name out_null.csv.)

I would think that the following should work for your requirement:
tRowGenerator_1 --> tMSSqlSP_1 --> tParseRecordSet_1 --> tFlowToIterate -iterate-> tFixedFlowInput --> tFileOutputDelimited_1  --> tLog
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

And it did. Thank you very much uInfinity. FYI, I couldn't use row1.FileNameCol, I had to use tFlowToIterate_1.FileNameCol, which got converted to ((String)globalMap.get("FileNameCol")) - Is this correct?

Many thanks again.
One Star

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

How you did it sounds better.
The components reference guide for tFlowToIterate mentions that global variables are set:
"Function: tFlowToIterate iterates on the input data and generates global variables."
Your way is more in line with the example for tFlowToIterate:
"Scenario: Transforming data flow to a list"
Five Stars

Re: [resolved] How to write every row from tParseRecordSet to a different delim file?

Hi dministrator
you can put tJava after tFlowToIterate then you will get Main connector to connect with tFixedFlowInput.
tFlowToIterate ----tJava--main--tFixedFlowInput
hope this will work.