Preprocessing of missing values using robust association rules

Abstract

Many of analysis tasks have to deal with missing values and have developed specific and internal treatments to guess them. In this paper we present an external method for this problem to improve performances of completion and especially declarativity and interactions with the user. Such qualities will allow to use it for the data cleaning step of the KDD1 process[6]. The core of this method, called MVC (Missing Values Completion), is the RAR2 algorithm that we have proposed in [14]. This algorithm extends the concept of association rules[l] for databases with multiple missing values. It allows MVC to be an efficient preprocessing method: in our experiments with the c4.5[12] decision tree program, MVC has permitted to divide, up to two, the error rate in classification, independently of a significant gain of declarativity.