One Star

Data quality audit

Hi,
We have a DB with more than 1000 tables and the client is reporting serious data integrity violations. There could be redundancy and other quality issues. This DB does not have a single FK constraint as they are all enforced by the application in some way.
As part of a technical audit I have to profile this DB. I am looking for good data quality tutorials. The only link I found is invalid now(http://www.talend.com/blog/2008/10/08/excellent-data-profiling-tutorial).
Is there a way to profile this DB even at a cursory level in a week and report quality issues ? We will fix them in the next step. After profiling we have to build a er diagram with proper constraints.
Appreciate any immediate assistance. I have Open studio.

Thanks,
Mohan
3 REPLIES
Seventeen Stars

Re: Data quality audit

For Open Studio there is no build-in solution solving your problem out of the box. But there are a lot of possibilities. I have made a system in which quality rules described as SQL code are saved in a database table. My customer could enhance these rules and a automated Talend job iterated through this rules, executing the SQLs and creates a report as text file about success or violation of a rule.
If you save the result also in database table, you can provide a report to your customer about the quality of your data.
I would add to every table, which should be checked a field, which marks a dataset as ok or not ok.
One Star

Re: Data quality audit

Does it mean that the Open version cannot be used to run a quality analysis ?
Thanks.
Two Stars

Re: Data quality audit

Hello jlolling,

 

I am very new to Talen Open Studio. I love the solution you have proposed. Are you able to share with me more details?

 

I need to write data quality rules some of which have many conditions. Any assistance is much appreciated.

 

Thank you.

Best Regards,

Lorna