Eliminate matching data from the file

Eight Stars

Eliminate matching data from the file

I have below source file and data from table A , if file contains any of the values from Table A then eliminate it from the file .. need help in either Java or anything

 

One logic i thought is convert entire row from file as string into Java so that i can compare and mark true or false and then filter , but need help to write java code

if any better logic please let me know 

 

Source file as below (Dynamic columns positions are not known)

-----------------------------

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
3486223^2018-10-22^^^34^2018-10-22 00:59:05^2018-10-22 ^ln-01@euf.com^708548590^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

9642532^2018-10-22^^84441111^607^2018-10-22 00:59:12^2018-10-22 ^acboid^346543245^webv

 

 

Table A i have below values 

708548590

965354999

346543245

 

So I want the output as 

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

 

Fifteen Stars TRF
Fifteen Stars

Re: Eliminate matching data from the file

This is a very common and simple use case.

Use a tFileInputDelimited to read the input file with "^" as a field separator.

Then with a tMap join the flow with the lookup table to exclude undesired lines ("inner join" + catch "inner join reject" options).

 


TRF
Eight Stars

Re: Eliminate matching data from the file

On what basis the joins be mentioned , as we dont know the position of this column has the value 

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Download