Add incremental number to output file

Five Stars

Add incremental number to output file

Hi everyone,

 

is it possible to get outfiles with an incremental number?

 

Let's say we have a job and this job will create the file output_001.csv

If we start the job again then we'll get output_002.csv and if we start the job 3rd time then output_003.csv etc.

 

Thanks for any advice.

Employee

Re: Add incremental number to output file

Hi,

 

    If you are storing the last file number in a location (it can be file, DB etc), you can read it from there to a context variable (say running_number).

 

     While writing the file, you can add the file name in the output as "output_"+context.running_number+".csv"

 

    Once the data output to file is complete, you can write the new output back to the area where you are storing last output file number. In this way, it will work fine. I would suggest to use a table in DB to store this control information.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

 

 


Warm Regards,
Nikhil Thampi
Please appreciate our members by giving Kudos for spending their time for your query. If your query is answered, please mark the topic as resolved :-)
Nine Stars

Re: Add incremental number to output file

Below  Approach  would help

 

1) keep a File having last run sequence in your local disk . Read this file at the start of job and use it to create your next run file. Update the same file for next run sequence.

(Note : If you are using Context file , you can simply update new sequence in that file)

 

2) You could use system Variable instead of File . Get/Set using value tSystem and tSetenv 

 

3) using tFileFetch , get the latest created file name in the output folder .

    Get the last part from string so for output_003.csv it would be 003 

    Add 1 to it , use new value to create Output.

 

 

Regards
Abhishek KUMAR

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads