Split into multiple files based on Date Value from Source

Seven Stars

Split into multiple files based on Date Value from Source

need help to achieve that We are source records with different dates  and want to split into different files based on the month of those dates 

 

Example 

I have input data from source table as below

 

 

Col1 Col2 Col3
--------------------------
abc newyork  2018-10-15
abc newjersey 2018-10-12
abc newyork   2018-11-15
abc newyork   2018-12-06

Output should have 3 files 

 

 

one for Month 201810

 

abc newyork  2018-10-15
abc newjersey 2018-10-12

one for Month 201811

abc newyork   2018-11-15

one for Month 201812

abc newyork   2018-12-06

 

Six Stars

Re: Split into multiple files based on Date Value from Source

read file --> tjavarow ( set global var like filenameYYYYmm with yyyymm of last col --> output file ( filename should be globalvar (filenameYYYYmm ) append mode.

this should work
Regards
Abhishek KUMAR
Thirteen Stars

Re: Split into multiple files based on Date Value from Source

I afraid - this is will not work

filename defined before subjob start

 

need insert tFlowToIterate which could define and filename and store row into memory (hash or variables) and 2nd subjob - read data and append them to file 

-----------
Six Stars

Re: Split into multiple files based on Date Value from Source

Thanks @Vapulkov ,

you are right . I did not thought about it.

I was ablr to do it with below steps

Read file -- > add col4 which in YYYYMM format of Col3(TjavaRow) --> file uniq on col4 in ( tUniqRow) -- > tflowtoItrate( put col4 in GlobalVar) --> read file again --> tFilterRow on GlobalVar --> tFilerOutPut ( Filename based on globalVar)

we could use tHashINput/output if we do not want to read files multiple time.
Regards
Abhishek KUMAR
Six Stars

Re: Split into multiple files based on Date Value from Source

TalendSolution.JPG

Regards
Abhishek KUMAR

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.