[resolved] Dynamic filename in tFileOutputDelimited component

One Star

[resolved] Dynamic filename in tFileOutputDelimited component

Hi,
I need help trying to dynamically specify the name of the file to create in tFileOutputDelimited. I have attached some images showing a simplified job:
tFileInputDelimited --> tMap --> tFileOutputDelimited
When I use the value of a field that's flowing into tMap (row1.fileName), I get only null value for the variable. So, in my example, when the contents of the input file is
file1,content1
file2,content2
I get the output filename as <full path>dynamicFileNamingOutputnull.txt. I expected to get two files:
dynamicFileNamingOutputfile1.txt, and
dynamicFileNamingOutputfile2.txt
Any help will be greatly appreciated.
Thanks,
Yuan

Accepted Solutions
One Star

Re: [resolved] Dynamic filename in tFileOutputDelimited component

ok - I figured out why I was getting a single line. I didn't check the "append" option in tFileOutputDelimited.
I'm all set.
Thanks,
Yuan

View solution in original post


All Replies
Community Manager

Re: [resolved] Dynamic filename in tFileOutputDelimited component

Hello
You should use tFlowToIterate+tFixedFlowInput to iterate each row, your job looks like:
tFileInputDelimited--row1---tFlowToIterate--iterate---tFixedFlowInput--main--tMap--tFileOutputDelimited
On tFixedFlowInput, generate one row of input flow, eg: define two fields on the schema: filename and content
set the value of filename column as: (String)globalMap.get("row1.filename")
set the value of content column as: (String)globalMap.get("row1.content")
on tFileOutputDelimited, set dynamically the file name like:
"D:/file/"+(String)globalMap.get("row1.filename")+".txt"
About usage of tFlowToIterate component, see 5827.
Best regards
shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] Dynamic filename in tFileOutputDelimited component

Thanks Shong for the quick response.
I did revise the flow as you suggested. However, I'm only getting a single (last) line of output when there are supposed to be multiple lines.
Input:
file1,content1_1
file1,content1_2
file1,content1_3
file2,content2_1
file2,content2_2
I expected to see
file1
content1_1
content1_2
content1_3
file2
content2_1
content2_2
However, I'm getting
file1
content1_3
file2
content2_2
It appears the Number of lines in tFixedFlowInput is the culprit. Am I missing something?
Thanks,
Yuan
One Star

Re: [resolved] Dynamic filename in tFileOutputDelimited component

I also tried using the tRowGenerator and the result is the same - only a single line (from the last row) is output in each file.
I expected it to generate a row for each incoming row. What am I missing?
Thanks for your help,
Yuan
One Star

Re: [resolved] Dynamic filename in tFileOutputDelimited component

ok - I figured out why I was getting a single line. I didn't check the "append" option in tFileOutputDelimited.
I'm all set.
Thanks,
Yuan

View solution in original post

Eight Stars

Re: [resolved] Dynamic filename in tFileOutputDelimited component

Hi shong

@shong 

the same iam trying in my job but at creation of my filename dynamically, it is appending zero at last

pfscreenshot

output filename should be

dfsdmay161

but am getting

dfsdmay1610

Regards

rekha

 

Six Stars

Re: [resolved] Dynamic filename in tFileOutputDelimited component

Hi @bhagyarekha,

 

Please uncheck the split output file in several files in advanced settings in tFileOutputDelimited component.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog