Split into multiple files based on Date Value from Source

Eight Stars

Split into multiple files based on Date Value from Source

need help to achieve that We are source records with different dates  and want to split into different files based on the month of those dates 

 

Example 

I have input data from source table as below

 

 

Col1 Col2 Col3
--------------------------
abc newyork  2018-10-15
abc newjersey 2018-10-12
abc newyork   2018-11-15
abc newyork   2018-12-06

Output should have 3 files 

 

 

one for Month 201810

 

abc newyork  2018-10-15
abc newjersey 2018-10-12

one for Month 201811

abc newyork   2018-11-15

one for Month 201812

abc newyork   2018-12-06

 


Accepted Solutions
Highlighted
Eleven Stars

Re: Split into multiple files based on Date Value from Source

TalendSolution.JPG

Regards
Abhishek KUMAR

All Replies
Eleven Stars

Re: Split into multiple files based on Date Value from Source

read file --> tjavarow ( set global var like filenameYYYYmm with yyyymm of last col --> output file ( filename should be globalvar (filenameYYYYmm ) append mode.

this should work
Regards
Abhishek KUMAR
Forteen Stars

Re: Split into multiple files based on Date Value from Source

I afraid - this is will not work

filename defined before subjob start

 

need insert tFlowToIterate which could define and filename and store row into memory (hash or variables) and 2nd subjob - read data and append them to file 

-----------
Eleven Stars

Re: Split into multiple files based on Date Value from Source

Thanks @Vapulkov ,

you are right . I did not thought about it.

I was ablr to do it with below steps

Read file -- > add col4 which in YYYYMM format of Col3(TjavaRow) --> file uniq on col4 in ( tUniqRow) -- > tflowtoItrate( put col4 in GlobalVar) --> read file again --> tFilterRow on GlobalVar --> tFilerOutPut ( Filename based on globalVar)

we could use tHashINput/output if we do not want to read files multiple time.
Regards
Abhishek KUMAR
Highlighted
Eleven Stars

Re: Split into multiple files based on Date Value from Source

TalendSolution.JPG

Regards
Abhishek KUMAR

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog