I have a requirement to create sequence in talend.
Basically records are coming from a source file.
for each source row i want to create a unique number.
here is where it gets complicated.
when a new file comes next day , the talend should pick the last generated number and then increment it with 1.
today the last generated sequence number is 100.
tomorrow the talend should generate sequence number from 100 . i.e. 101,102,103,104.....
This means talend should keep the history of previously generated last sequence number.
So, in such a case you have to persist this last sequence value somewhere, in the target database (if any) or in a dedicated file.
If the records are stored in a database, you can also get the max value from the corresponding field using the appropriate Select.
When you got the desired value, you need to store it in a global variable, then reuse this variable to initialize the sequence with something like:
Numeric.sequence("yourSequence", (Integer)globalMap.get("yourGlobal"), 1)
Hope this helps.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Learn how to make your data more available, reduce costs and cut your build time
Read about OTTO's experiences with Big Data and Personalized Experiences
Take a look at this video about Talend Integration with Databricks