Fuzzy rule-based framework for medical record validation

Abstract

Data cleaning is an important part of the knowledge discovery process. The principal causes of data anomalies include incomplete information, absence of a unique identifier across multiple databases, inconsistent data, existence of data entry errors and logically incorrect data. This situation is further exacerbated while integrating data from multiple, disparate data sources. Since data quality is directly related with the quality of services in data-driven applications, such as medical informatics, a reliable data cleaning solution, which allows rapid and precise detection of invalid data, is needed. Most existing data cleaning solutions are domain specific, time-consuming and do not easily accommodate logical validations. In this paper, we propose a Fuzzy rulebased framework, which is domain independent, flexible and easily accommodates physical as well as logical validations. We have implemented existing cleaning strategies (i.e. Sorted Neighborhood Method), and enhanced them by using state-of-the-art algorithms (i.e. Rete, Bigram). As proof-ofconcept, our prototype system was applied to real patient data. Experimental results illustrate that our framework is extensible and allows rapid detection of invalid data with high precision.