Eliminate matching data from the file

Seven Stars

Eliminate matching data from the file

I have below source file and data from table A , if file contains any of the values from Table A then eliminate it from the file .. need help in either Java or anything

 

One logic i thought is convert entire row from file as string into Java so that i can compare and mark true or false and then filter , but need help to write java code

if any better logic please let me know 

 

Source file as below (Dynamic columns positions are not known)

-----------------------------

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
3486223^2018-10-22^^^34^2018-10-22 00:59:05^2018-10-22 ^ln-01@euf.com^708548590^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

9642532^2018-10-22^^84441111^607^2018-10-22 00:59:12^2018-10-22 ^acboid^346543245^webv

 

 

Table A i have below values 

708548590

965354999

346543245

 

So I want the output as 

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

 

Highlighted
Forteen Stars TRF
Forteen Stars

Re: Eliminate matching data from the file

This is a very common and simple use case.

Use a tFileInputDelimited to read the input file with "^" as a field separator.

Then with a tMap join the flow with the lookup table to exclude undesired lines ("inner join" + catch "inner join reject" options).

 


TRF
Seven Stars

Re: Eliminate matching data from the file

On what basis the joins be mentioned , as we dont know the position of this column has the value 

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.