Hi team, Now, I got two sources by tMSSqlInput and joined them in the tMap. I encountered a performance issue when i ran the subjob. The issue is which need to get all data than go to join. I think it's very slow. I want to do the filter in the tMSSqlInput, but i am not sure whether i can do it or not. Or do you have any idea for the issue?
Hi, The tMap component needs load all data into local firstly and then doing match. Depending on the filters you are using you could use ELT components to let the DBMS handle the filtering. Have you already checked talend ELT components for sql server? tELTMSsqlInput, tELTMSSqlOutPut and tELTMSSqlMap. Best regards Sabrina
-- Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Filter in tMySQLInput - it is normal WHERE in SQL code - so this is You can do easily, but main idea of ELT - load data to destination and than compare and transform ok, You are have 50 M in Lookup table, but how many in new(LoadPackage...)?