[resolved] Get input file row by row as a string

One Star aca
One Star

[resolved] Get input file row by row as a string

Hi, I apologize if this have been answered before.
TOS 3.2.0, Java
I have simple job: Get the list of files from a directory and parse one by one and store data in database table:
tFileList => tFileInputDelimited => tMap => Database Table
Each file row has more than 100 ~ delimited values, and each file has sometimes 10000 rows.
I use tMap to extract few of values in each row and map to a field in database table.
One of the values is file name itself which I got like this: ((String)globalMap.get("tFileList_1_CURRENT_FILE"))
One of the field in the table is current file row itself. Since I have more that a 100 delimited columns it would be a pain to concatenate each individual column. Is there a variable that I can use to reference entire row (as a string) as it comes from the file.
Thanks in advance.

Accepted Solutions
Community Manager

Re: [resolved] Get input file row by row as a string

Hello
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput
On tFixedFlowInput, only generate one row with one column, set the value of column as: (String)globalMap.get("row1.line")
On tExtractDlimitedFields, extract the delimited values
On tmap, add a new column called line, set the expression as: (String)globalMap.get("row1.line")
if you need to filter some columns, doing that on another tmap or tFilterColumns component.
about tFlowToIterate, see 5827.
Best regards

shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

All Replies
Community Manager

Re: [resolved] Get input file row by row as a string

Hello
One of the field in the table is current file row itself. Since I have more that a 100 delimited columns it would be a pain to concatenate each individual column.

You can use tFileInputFullRow component to read the whole line as one row.
Best regards

shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star aca
One Star

Re: [resolved] Get input file row by row as a string

Thanks Shong, but I also need to extract few elements from that line.
tFileInputFullRow let me extract just whole row and tfileInputDelimited let me extract delimited values that I need.
What I need to map is 3 elements from file delimited row and a whole row itself. I there a way to do that. is there variable in tfileInputDelimited that holds whole row string.
Thanks.
Community Manager

Re: [resolved] Get input file row by row as a string

Hello
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput
On tFixedFlowInput, only generate one row with one column, set the value of column as: (String)globalMap.get("row1.line")
On tExtractDlimitedFields, extract the delimited values
On tmap, add a new column called line, set the expression as: (String)globalMap.get("row1.line")
if you need to filter some columns, doing that on another tmap or tFilterColumns component.
about tFlowToIterate, see 5827.
Best regards

shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star aca
One Star

Re: [resolved] Get input file row by row as a string

perfect, thanks....
One Star

Re: [resolved] Get input file row by row as a string

Dear Shong,
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput in this flow if I have to extract data for fixed width file for each row with different length, which component i should use in place of tExtractDlimitedFields?
Community Manager

Re: [resolved] Get input file row by row as a string

Dear Shong,
tFileInputFullRow ---row1-->tFlowToIterate--iterate-->tFixedFlowInput--->tExtractDlimitedFields--tMap---tMysqlOutput in this flow if I have to extract data for fixed width file for each row with different length, which component i should use in place of tExtractDlimitedFields?

Take a look at tExtractPositionalFields component.
Regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business