How to update prev row after comparing with current and prev row?

Highlighted
Six Stars

How to update prev row after comparing with current and prev row?

Hi All,

 

I am new to TOS for data processing. My previous approach was with tUniqRow but changed it using tMemorizeRow and tJavaFlex.

 

 I am using tMemorizeRows and tJavaFlex for my duplicate row check.

Using tMemorizeRows  with 2 row count, I am checking 2 row at a time. Based on my one condition, I need to update one column in the prev row.

So, in tJavaFlex based on my memorizerow data, I am able to compare and update a column for my current row in context. But to update a column value for my previous row (matched row by condition using memorizerow), how can I do that? How can I get previous row to update any column with new value.

 

I tried with tJavaFlex_2/tJavaRow after my tJavaFlex_1 but it executes one row at a time. So, I am not able to get previous row. Is there anyway, I can search the whole result row object(tJavaFlex_1) and set value?  Can I use tMap to solve the problem? Thanks in advance.

 

Capture.JPG


Accepted Solutions
Highlighted
Community Manager

Re: How to iterate uniques and duplicates of tUniqRow for a condition

It's probably best that use the tHashInput and tHashOutput components for this.

https://help.talend.com/reader/8_lXEInq2ehdXds6XE9PsA/OxFrHZZyW9fvSE5SUBJOKQ

https://help.talend.com/reader/8_lXEInq2ehdXds6XE9PsA/kQWU35YXytKECckZ5EqvGg

 

Read the data into a tHashOutput for each row (uniques/non-uniques), then use the tHashInput components to read that data in again and join using a tMap.

 

View solution in original post

Nine Stars

Re: How to iterate uniques and duplicates of tUniqRow for a condition

@dattasud , Yeah with the currrent design, the hashinput subjob is executed at the same time as the salesforce connection. You have to connect the HashInput subjob on 'On subjob ok' trigger from tSalesforceInput

View solution in original post


All Replies
Highlighted
Community Manager

Re: How to iterate uniques and duplicates of tUniqRow for a condition

It's probably best that use the tHashInput and tHashOutput components for this.

https://help.talend.com/reader/8_lXEInq2ehdXds6XE9PsA/OxFrHZZyW9fvSE5SUBJOKQ

https://help.talend.com/reader/8_lXEInq2ehdXds6XE9PsA/kQWU35YXytKECckZ5EqvGg

 

Read the data into a tHashOutput for each row (uniques/non-uniques), then use the tHashInput components to read that data in again and join using a tMap.

 

View solution in original post

Highlighted
Six Stars

Re: How to iterate uniques and duplicates of tUniqRow for a condition

Thanks for your input @rhall_2_0 .

 

I changed my approach for the problem. I am using tMemorizeRows and tJavaFlex for my duplicate row check.

Using tMemorizeRows  with 2 row count, I am checking 2 row at a time. Based on my condition, I need to update one column in the row.

So, in tJavaFlex based on my memorizerow data, I am able to update a column for my current row in context. But to update a column value for my previous row (matched row by condition using memorizerow), how can I get previous row to update any column with new value.

 

I tried with tJavaFlex_2/tJavaRow after my tJavaFlex_1 but it executes one row at a time. So, I am not able to get previous row. Is there anyway, I can search the whole result row object(tJavaFlex_1) and set value?  Can I use tMap to solve the problem? Thanks in advance.

Capture.JPG

 

Highlighted
Community Manager

Re: How to iterate uniques and duplicates of tUniqRow for a condition

The method you are looking for (to search the whole result rows) with a tMap is essentially what I was describing in my first suggestion. I am not sure how your new method will work if I am honest. Have you tried using the tHash components?

Highlighted
Six Stars

Re: How to iterate uniques and duplicates of tUniqRow for a condition

Hi @rhall_2_0 

 

Yes, I checked the tHashOutput and tHashInput components and used it like below flow. 

I am getting runtime exception with tHashInput_2 that its not initialized. The  tHashInput_2 is linked to tHashOutput_2 which is having the duplicate rows.

I wrote condition in tMap for my duplicate row checks and updating the value in rows. 

 

[statistics] connecting to socket on port 3722
[statistics] connected
Exception in component tHashInput_2 (Copy_of_AccountStatusChange)
java.lang.RuntimeException: The hash is not initialized : The hash must exist before you read from it
at stateofbusiness.copy_of_accountstatuschange_0_1.Copy_of_AccountStatusChange.tHashInput_2Process(Copy_of_AccountStatusChange.java:2601)
at stateofbusiness.copy_of_accountstatuschange_0_1.Copy_of_AccountStatusChange.tHashInput_1Process(Copy_of_AccountStatusChange.java:1511)
at stateofbusiness.copy_of_accountstatuschange_0_1.Copy_of_AccountStatusChange.runJobInTOS(Copy_of_AccountStatusChange.java:5655)
at stateofbusiness.copy_of_accountstatuschange_0_1.Copy_of_AccountStatusChange.main(Copy_of_AccountStatusChange.java:5504)
[statistics] disconnected

 

The tHashInput_2 is getting initiated from tHashInput_1. And also the this right side block is getting complied first then the actual main left side job. Is there any way to correct the flow. The flow should go from left to right.

 

Capture.JPG

 

Nine Stars

Re: How to iterate uniques and duplicates of tUniqRow for a condition

@dattasud , Yeah with the currrent design, the hashinput subjob is executed at the same time as the salesforce connection. You have to connect the HashInput subjob on 'On subjob ok' trigger from tSalesforceInput

View solution in original post

Highlighted
Six Stars

Re: How to iterate uniques and duplicates of tUniqRow for a condition

@nivedhitha Thanks for your input. The error is resolved.
Thanks @rhall_2_0 for your help and time.
Highlighted
Community Manager

Re: How to iterate uniques and duplicates of tUniqRow for a condition

Glad you got it sorted, and thanks to @nivedhitha for helping to resolve the usage problem

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now