Four Stars

How to get next record's value while processing an input ?

 
10 REPLIES
Twelve Stars

Re: How to get next record's value while processing an input ?

I believe this tutorial will help with this.....

 

https://www.rilhia.com/quicktips/quick-tip-compare-row-value-against-value-previous-row

 

If you want to "look ahead" you simply adjust this slightly to hold processing the first row until the second has arrived. Thus allowing you to appear to "look ahead"

Rilhia Solutions
Four Stars

Re: How to get next record's value while processing an input ?

Can you please explain it in detail ? how is it possible to get the next row ?

 

Thanks ,

Praveen.

Twelve Stars

Re: How to get next record's value while processing an input ?

It is not possible for the current row to get the "next" as such. So what I am suggesting is using a variation on what that tutorial shows. 

 

When your job reads the first row, it is essentially stored in memory. Then, when the second row arrives, the first row does what it needs to with the second row's data, is released and then the second row is stored in memory. This goes on throughout the job until you get to the end of the data set.

 

This is enabled by the fact that the tMap variables are processed from top to bottom and are held between data rows. Using this functionality (as described in the tutorial....you can literally follow it and recreate it in an example job to see it happening), you can do this inter-row calculation.

Rilhia Solutions
Four Stars

Re: How to get next record's value while processing an input ?

In my scenario , current row's data will be modified as per next row's data .So as per your suggestion if i store the first row data in a variable and wait for the second row to arrive , i can compare that with the variable value but how can i edit the previous row data based on this comparison ?

Thanks ,

Praveen . 

Twelve Stars

Re: How to get next record's value while processing an input ?

The previous row's data is in memory. You can edit it in memory as well.

When you output the variables that the row columns are stored in, you can modify the data there....or do it in another tMap variable. 

 

This isn't *easy*, but it is possible and quite logical when you get your head into it. Try it out. The best way to learn something like this is to test it in an example job.

 

Rilhia Solutions
Four Stars

Re: How to get next record's value while processing an input ?

HI Rhall,

 

I referred the link you have provided but getting error.

My requirement is to store previous record hashkey  and price value in two variables. if hashkey is same calculate price difference as below

input

hash key | postcode | Price

1000           abc             100,000

1000           abc             200,000

1000           abc             250,000

1001            xyz               50,000

1001            xyz             250,000

 

expected:

 

hash key | postcode | Price        |   profitmade

1000        |   abc         |    100,000  |              0

1000        |   abc         |   200,000   |  100,000

1000        |   abc        |    250,000   |    50,000

1001         |   xyz         |      50,000   |             0

1001         |   xyz          |   250,000   |   200,000

 

Error I am getting:

org.talend.designer.runprocess.ProcessorException: Job compile errors
At least job "testfile" has a compile errors, please fix and export again.
Error Line: 1275
Detail Message: v_Prev_Price cannot be resolved to a variable

I have attached the print screen of job.

variable v_price expression: 

(v_PrevHash!=null && v_PrevHash == row1.hashkey)?(row1.Price-v_Prev_Price):row1.Price

 

Can you please check what could be the issue?

 

I also searched and found that there is component 'tMemorizeRows'  I am not able find this component in my talend studio.

'tMemorizeRows' :  https://help.talend.com/reader/mjoDghHoMPI0yuyZ83a13Q/0oEBI6YCTFXSgc5Z9Gpr6Q 

Twelve Stars

Re: How to get next record's value while processing an input ?

OK, when using tMap variables, they are referred to using Var. before the name. So your variables should be named like this Var.v_Prev_Price, for example. You can drag and drop your tMap variables to another tMap variable expression if you like. That should show you how the names should be formatted. This is almost certainly the cause of your Java error.

Rilhia Solutions
Twelve Stars TRF
Twelve Stars

Re: How to get next record's value while processing an input ?

Hi,

In my opinion, the simplest is to use a tJavaRow instead of a tMap for this case.

It should look like this:

// Initialize output_row fields from input_row fields values
output_row.hashkey = input_row.hashkey;
output_row.postcode = input_row.postcode;
output_row.price = input_row.price;

// Set ProfitMade depending on comparison result of previous and current hashkey
if((Integer)globalMap.get("PrevHash") != null && (Integer)globalMap.get("PrevHash") == input_row.hashkey)
	output_row.ProfitMade = input_row.Price - (Integer)globalMap.get("PrevPrice");
else
	output_row.ProfitMade = 0;

// Set the global variables values for next iteration
globalMap.set("PrevHash", input_row.hashkey);
globalMap.set("PrevPrice", input_row.Price);

Check field names and datatypes (integer vs float) before to try this in your job.

Hope this helps.


TRF
Four Stars

Re: How to get next record's value while processing an input ?

Thank rhall, I  added 'Var' before using variable and it worked. but facing datatype incompatible issue now.

hashkey field data  is being generated using MD5 function(which is basically hexadecimal). My source is file. talend reading hashkey as string  and giving below error datatype defined in talend is int ( I tried changing datatype as int, long etc)

 

[statistics] connecting to socket on port 3446

[statistics] connected

For input string: "hashkey"

For input string: "ddbfcac5d5bb83f97906b3326e4dfd97"

For input string: "ddbfcac5d5bb83f97906b3326e4dfd97"

For input string: "ddbfcac5d5bb83f97906b3326e4dfd97"

For input string: "ddbfcac5d5bb83f97906b3326e4dfd97"

For input string: "ddbfcac5d5bb83f97906b3326e4dfd97"

[statistics] disconnected

Job testfile ended at 14:17 01/11/2017. [exit code=0]

Twelve Stars

Re: How to get next record's value while processing an input ?

Can you show some screenshots of your job configuration, since I do not understand what you are trying to do. Are you reading in an MD5 String? I'll need a bit more info

Rilhia Solutions