Talend duplicating records

Four Stars

Talend duplicating records

Hi guys.

My talend job is pulling a file in and breaking down the data into records, however on the input_row the data is coming in fine but the output_row is duplicating the data.

 

the following is my tjavarow code:

 

int recordcount = ((int)globalMap.get("readrecords"));

int validcounter = ((int)globalMap.get("validcounter"));
int invalidcounter = ((int)globalMap.get("invalidcounter"));


System.out.println(input_row.toString());
if (input_row.Broken_Name_Flag==1) { // 1 = the row is fine

globalMap.put("validcounter", validcounter + 1);

output_row.Source = input_row.Source;
output_row.UCN = input_row.UCN;
output_row.Broken_Name_Flag = input_row.Broken_Name_Flag;
output_row.Customer_Title = input_row.Customer_Title;
output_row.Customer_Name_s = input_row.Customer_Name_s;
output_row.Customer_Middle_Names = input_row.Customer_Middle_Names;
output_row.Customer_Surname = input_row.Customer_Surname;
output_row.Customer_Full_Name = input_row.Customer_Full_Name;
output_row.Gender = input_row.Gender;
output_row.ID_Number = input_row.ID_Number;
output_row.Date_of_Birth = input_row.Date_of_Birth;
output_row.Citizenship = input_row.Citizenship;
output_row.Nationality = input_row.Nationality;
output_row.Country_of_Residence = input_row.Country_of_Residence;
output_row.Address_Line_1 = input_row.Address_Line_1;

}
else {
globalMap.put("invalidcounter", invalidcounter + 1);
}

output_row.validcount= validcounter + 1;
System.out.println(output_row.toString());
output_row.invalidcount= invalidcounter + 1;

 

the following is my output:

 

Starting job RiskRatingFileUpload at 15:17 12/04/2018.


[statistics] connecting to socket on port 3677
[statistics] connected
[trace] connecting to socket on port 4614
[trace] connected
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=null]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=0,Customer_Title=Dr,Customer_Name_s=Adriaan,Customer_Middle_Names=,Customer_Surname=,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Mar 02 00:00:00 CAT 1961,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Plot 56, 121 Swallow Street, Doringpoort, 0159"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=0,Customer_Title=Mr,Customer_Name_s=Craig,Customer_Middle_Names=Warrick,Customer_Surname=Lyall,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Jan 03 00:00:00 CAT 1985,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 10, 44 ON Bantry, 3 Antrim Road| Bryanston, 2191"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Dr,Customer_Name_s=Martin,Customer_Middle_Names=Johannes Ulrich Thomas,Customer_Surname=van Wijngaarden,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Thu Dec 01 00:00:00 CAT 1960,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="5 Grieg Street, Vanderbijlpark, 1911",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Ashay,Customer_Middle_Names=,Customer_Surname=Deochand,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Tue May 03 00:00:00 CAT 1983,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 1 Umuzi, 136 Daisy Street, Sandown, 2196"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Ashay,Customer_Middle_Names=,Customer_Surname=Deochand,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Tue May 03 00:00:00 CAT 1983,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="Unit 1 Umuzi, 136 Daisy Street, Sandown, 2196",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row1Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Zeyn,Customer_Middle_Names=,Customer_Surname=Mahomed,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Sun Jan 08 00:00:00 CAT 1984,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="26A Central Street, Houghton Estate"]
riskratingfileupload.riskratingfileupload_0_1.RiskRatingFileUpload$row3Struct@effeb7e5[Source=Ashburton Principal Funds,UCN=,Broken_Name_Flag=1,Customer_Title=Mr,Customer_Name_s=Zeyn,Customer_Middle_Names=,Customer_Surname=Mahomed,Customer_Full_Name=,Gender=M,ID_Number=,Date_of_Birth=Sun Jan 08 00:00:00 CAT 1984,Citizenship=South Africa,Nationality=South Africa,Country_of_Residence=South Africa,Address_Line_1="26A Central Street, Houghton Estate",validcount=1,invalidcount=1]
Load jars loaded successfully!
Connected to TIBCO !
session not null !
[statistics] disconnected
[trace] disconnected
Job RiskRatingFileUpload ended at 15:17 12/04/2018. [exit code=0].

 

any help will be appreciated

Tags (1)
Nine Stars

Re: Talend duplicating records

If possible please provide screen shot of job.

 

Regards,

Veeru Boppudi
Four Stars

Re: Talend duplicating records

As requested

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads