Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Lookup on multiple columns

Highlighted
Six Stars

Lookup on multiple columns

Hi

I'm hoping someone can help.  

I have 3 columns in a mySQL input and need to make sure I only process those records I haven't seen before. 

My target database table contains the same 3 columns plus an integer id column and the table has a unique index on these 3 columns.  

I have pulled database table into the job as a lookup and am about to join the 3 input columns to their matching column in the lookup.  Where no match occurs and my id field is 0 then I can move forward with the record for processing through to the database table.

 

 

My concern is that I'm sure I've read somewhere that this isn't the best approach so wanted to reach out to the community to get some thoughts on how I should approach it.

Thanks

Dave


Accepted Solutions
Highlighted
Six Stars

Re: Lookup on multiple columns

Dave,

You can do this many ways. one easy approach is to use tMap and use Inner join. join all the 3 columns. In the output table, go to tMap settings and choose "catch lookup inner join reject" as true.  This approach would work perfectly fine for 2 table join.

 

In case of more than 3 table joins, use the conventional left outer join and select the id column from right table and check for null.

 

Regards,

Ragu

View solution in original post


All Replies
Highlighted
Six Stars

Re: Lookup on multiple columns

Dave,

You can do this many ways. one easy approach is to use tMap and use Inner join. join all the 3 columns. In the output table, go to tMap settings and choose "catch lookup inner join reject" as true.  This approach would work perfectly fine for 2 table join.

 

In case of more than 3 table joins, use the conventional left outer join and select the id column from right table and check for null.

 

Regards,

Ragu

View solution in original post

Highlighted
Six Stars

Re: Lookup on multiple columns

Ragu

Thanks - this worked a treat!

 

Thanks

Dave

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog