Rejecting invalid data records and write bad records to a file
The need is to import CSV files and load the data into database and write all bad/invalid data to a separate csv file. My question is what is the best way to do this? Based on my understanding, I can either use "Trigger-> On Component Error" or use tMap. Just wonder what is a better approach between these two or is there any other approach. Thanks.
Re: Rejecting invalid data records and write bad records to a file
How are you determining the validity or your data? Talend offers several components you could use. For example there is a tSchemaComplianceCheck component. "Validates all input rows against a reference schema or checks type, nullability, length of rows against reference values. The validation can be carried out in full or partly."