as I mentioned earlier, it still keep inserting records even if i set it to "Insert". following is the example data: ID Name SRC ------------------------------------ 81687 text1 source 81686 text1 source 81685 text1 source
@janhess > I meant the GlobalKey Map variable. By the way: i'm having the same thing as @infoleather mentioned. rows are rejected when I run the job for second or third time. But for the first time, it is inserting everything... that means duplication is still there!
globalMap.Get items are setup in tMap so can be used in the Query, to find the exact match. don't get what you mean? "The table you are looking for duplicates in is keyed on an id so you would need to look for duplicate data items." please be descriptive!
Well that obviously doesn't work as you're not finding duplicates. When you output to your lookup table you will generate duplicates as it is keyed on an auto generated id so you will never get duplicate keys.
my concern is not the duplicate keys, but the other attributes of the table. please check the screenshot of mySQL results, you will see there are duplication and I don't want them to be repeatedly inserted.
1. How can I make the key fields? 2. That's why I wrote the SQL Query with a lot of GlobalMap.Get variables to check if they exists.. but after that I don't know how to proceed? please let me know on both of the solutions how to implement?