Eliminate matching data from the file

Eight Stars

Eliminate matching data from the file

I have below source file and data from table A , if file contains any of the values from Table A then eliminate it from the file .. need help in either Java or anything

 

One logic i thought is convert entire row from file as string into Java so that i can compare and mark true or false and then filter , but need help to write java code

if any better logic please let me know 

 

Source file as below (Dynamic columns positions are not known)

-----------------------------

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
3486223^2018-10-22^^^34^2018-10-22 00:59:05^2018-10-22 ^ln-01@euf.com^708548590^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

9642532^2018-10-22^^84441111^607^2018-10-22 00:59:12^2018-10-22 ^acboid^346543245^webv

 

 

Table A i have below values 

708548590

965354999

346543245

 

So I want the output as 

7465721^2018-10-22^^^34^2018-10-22 00:59:01^2018-10-22 ^ln-01@abc.com^663986903^Auth
8975432^2018-10-22^^18459068^607^2018-10-22 00:59:12^2018-10-22 ^mmcvetril^185245600^OnlineAcc

 

Fifteen Stars TRF
Fifteen Stars

Re: Eliminate matching data from the file

This is a very common and simple use case.

Use a tFileInputDelimited to read the input file with "^" as a field separator.

Then with a tMap join the flow with the lookup table to exclude undesired lines ("inner join" + catch "inner join reject" options).

 


TRF
Eight Stars

Re: Eliminate matching data from the file

On what basis the joins be mentioned , as we dont know the position of this column has the value 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now