Four Stars

[resolved] How to get the last 10 lines from file using tsamplerow?

Hi Master,
I use tfileinputDelimited to read a file and then need to get the last 10 lines from this file.
i try to use tSamplerow but it can not read NB_LINE from tFileInputDelimited when using Main Connection.
i try to save it first to context but still return null when using Main connection.
Do you have any idea?
Thank you and best regards.
1 ACCEPTED SOLUTION

Accepted Solutions
Community Manager

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Hi 
The global var NB_LINE can be only used on another subjob, you need to read the source file two times (or one time if you want to store the data into memory). For example:
tFileInputDelimited_1--main--tJava_1
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tFileInputDelimited_2--main--tSampleRow--main--tLogRow
or 
tFileInputDelimite--main--tHashOutput
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tHashInput--main--tSampleRow--main--tLogRow
on tJava_2: get the NB_line, define the range expression for used later on tSampleRow
int nb_line=((Integer)globalMap.get("tFileInputDelimited_1_NB_LINE"));
int begin=nb_line-9;
context.range=begin+".."+nb_line;

//Define a context var called range in the job.
on tSampleRow, set the range expression with context var.
context.range
----------------------------------------------------------
Talend | Data Agility for Modern Business
4 REPLIES
Community Manager

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Add a sequence id to each row on tMap, and then sort the rows with desc model on tSortRow, define a var to generate the sequence id on another tMap, filter the first 10 lines using the sequence id.
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Hi shong thank you for your fast response,
Nice idea but a little bit inefficient i think because sorting large data will take much time.
It will be great if i can use NB_LINE that already provided by tFileInputDelimited.  Do you have any idea that i can do, maybe doing something with tSampleRow, so it can read the NB_LINE?
Thank you and best regards,
Community Manager

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Hi 
The global var NB_LINE can be only used on another subjob, you need to read the source file two times (or one time if you want to store the data into memory). For example:
tFileInputDelimited_1--main--tJava_1
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tFileInputDelimited_2--main--tSampleRow--main--tLogRow
or 
tFileInputDelimite--main--tHashOutput
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tHashInput--main--tSampleRow--main--tLogRow
on tJava_2: get the NB_line, define the range expression for used later on tSampleRow
int nb_line=((Integer)globalMap.get("tFileInputDelimited_1_NB_LINE"));
int begin=nb_line-9;
context.range=begin+".."+nb_line;

//Define a context var called range in the job.
on tSampleRow, set the range expression with context var.
context.range
----------------------------------------------------------
Talend | Data Agility for Modern Business
Four Stars

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Okay shong. i prefer your second solution..
it work for me.
Thank you shong.