Conditional dependencies: A principled approach to improving data quality

Abstract. Real-life date is often dirty and costs billions of pounds to businesses worldwide each year. This paper presents a promising ap-proach to improving data quality. It effectively detects and fixes inconsis-tencies in real-life data based on conditional dependencies, an extension of database dependencies by enforcing bindings of semantically related data values. It accurately identifies records from unreliable data sources by leveraging relative candidate keys, an extension of keys for relations by supporting similarity and matching operators across relations. In con-trast to traditional dependencies that were developed for improving the quality of schema, the revised constraints are proposed to improve the quality of data. These constraints yield practical techniques for data re-pairing and record matching in a uniform framework.