Almost resolved - How to create dynamic filename from input data value

Four Stars

Almost resolved - How to create dynamic filename from input data value

Hello everyone
I've been struggling to get a handle on context variables and dynamic file naming, reviewing other posts with little success up to a few moments ago... I'll put the description of my solution first, and the original "problem" post at the end.
The screen shot named "naming_resolved" shows the job that
1. Reads working hours data from an input source
2. Writes to a database
3. Writes to a dynamically named file. The file name is based on one of the input rows.
My original attempt was to use a tJavaFlex component to set a context variable to the file name. My solution uses (as is mentioned in other posts) tFlowToIterate and tFixedFlowInput.
The tFlowToIterage component has nothing for me to set.
In the tFixedFlowInput component I defined the metadata for the input stream (see the screen shot). This creates the variable (PAYDATE) that I need for my file name.
In the tFileOutputDelimited control, the file name is set to "Headcount" + "_" + StringHandling.LEFT((String)globalMap.get("SourceData.PAYDATE"),10) + ".csv"
Given my relatively low level of experience with TOS - perhaps this will help someone else.
- Russell

--- Original "Problem" Post ---
My goal is to dynamically name an output file based on the value (pay date) of a column read from an input stream. The value is repeated on all rows of the input stream. I'm selecting work hours data from the input source and outputting it to SQL Server in addition to the output file. The output file should be named as "Headcount_<paydate>_<generation date/time>.csv" in order to keep multiple versions separate. Each run is for only one pay date.
The attached screen shot is the overall job. It's not complicated.
In the tFileOutputDelimited component, I'm trying to set the file name as "Headcount" + "_" + (context.Period) + "_" + TalendDate.formatDate("yyyyMMddHHmmssSSS", TalendDate.getCurrentDate()) + ".csv"
The resulting file name is "Headcount_null_20100506104433709.csv"
Any help (and patience) will be appreciated.
Four Stars

Re: Almost resolved - How to create dynamic filename from input data value

The issue that this solution appears to face is that because we're now processing row-by-row, the generated delimited file is really created (appended) row-by-row, and the SQL Server table that I'm creating is also being done so row-by-row.
The "Append" setting must be set for the delimited file in order to create it, but then every time I run the job it appends the data instead of recreating the file.
Hmm... still more work to do.


Talend named a Leader.

Get your copy


Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables


Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema


Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables