How to generate target files dynamically at run time

Five Stars

How to generate target files dynamically at run time

Hi,
I have a source table/file with thousands of records, each record belongs one specific department, and number of department may grow.

and the requirement is to generate one file for each department and load corresponding data into that file, like that need to create files at run time depends on the number of department coming from the source.

Ex:

empid,name,deptid
100,a,10
200,b,20
300,c,10
400,d,40
500,c,30
600,e,10
700,f,40
800,g,20
as of now there are 4 different department from source file, so need to create 4 files at run time

FileName: file_10.txt File_20.txt
empid,name,deptid empid,name,deptid
100,a,10 200,b,20
300,c,10 800,g,20
600,e,10

like that need to create, and what are the components are required to implements this
Thanks

Tags (1)

Accepted Solutions
Sixteen Stars TRF
Sixteen Stars

Re: How to generate target files dynamically at run time

Does this help?
If so, thanks to mark your case as solved.

TRF

View solution in original post


All Replies
Forteen Stars

Re: How to generate target files dynamically at run time

@jpreddym, check the solution of @Mayur Gupta ,it works for you

 

 

https://community.talend.com/t5/Archive/Split-a-input-file-into-multiple-output-files/m-p/189235

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Tags (1)
Sixteen Stars TRF
Sixteen Stars

Re: How to generate target files dynamically at run time

tFileInputDelimited --> tFlowToIterate

|

(onSubjobOK)

|

tIterateToFlow --> tFileOutputDelimited

 

Use the following expression for the output filename :

"file_" + (Integer)globalMap.get("deptid") + ".txt"

TRF
Sixteen Stars TRF
Sixteen Stars

Re: How to generate target files dynamically at run time

Does this help?
If so, thanks to mark your case as solved.

TRF

View solution in original post

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog