How to store Variable while running Talend Job

One Star

How to store Variable while running Talend Job

hi,
Scenario->
                  I want to store Some record in File or table with ID column.
                 Suppose, For first run Storing 10 record
                 then, O/P will be 
ID
-----
1
2
3
.
.
10
         Now for Second run it should start from 11. for that we need to store max of last load i.e10
so it will be 
ID
---
1
2
3
.
.
10
11
12
13
...
how to do this in Talend.
Community Manager

Re: How to store Variable while running Talend Job

Hi
In the beginning of job, read the max ID from a static file and store it to a context variable or global variable for used later. For example:
tFileInputDelimimitecd--main--tJavaRow
on tJavaRow:
context.max_value=input_row.max_value;

In the main processing of job,
if source data are stored in database, select records only when ids bigger than max value, for example:
"select * from tableName where id>"+context.max_value
if source data are read from file or other data sources, filter the rows on tMap, and then load the result to destination. For example:
tFileInputDelimited--main--tMap--o/p component--main--tSortRow--tMap_2--main--tFleileOutDlimited_2
on tSortRow, sort all the ids with desc model.
on tMap_2: add a sequence number for each row, and filter the rows to get only the first row which is the max ID for this execution.
on tFleileOutDlimited_2: update the max ID to the static file.
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: How to store Variable while running Talend Job

Hi Shong,
Thanks for reply. I did the same but I am getting same value again not newly max value.
Regards,
Kiran B.
Community Manager

Re: How to store Variable while running Talend Job

Hi 
I don't know which step your are doing some things wrong, can you please export your job and send it to me?
BR
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog