I'm trying to load from multiple .CSV files into a MySQL source database. Below I'm attaching a sample snip of the data consisting of 7 columns:
Each record represents either a completed Job or Service order. I have four columns combined to be one primary key i.e 'JobID '+ 'Type' + 'Franchisee' + 'RoyaltyPostDate'.
When I'm trying to execute the job it hangs in the middle as duplicate entries are available. If seen in the screenshot 4th,5th and 6th row is the same job repeated thrice as they have different 'Installdate'. \
So, when I'm executing I'd like to load only one rows(i.e 4th record) and reject 5th,6th record and resume loading data again from 7th record.
Is there any possible way to do this. Any help would be appreciated.
Solved! Go to Solution.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend
Test drive Talend's enterprise products.