Data currency assessment through data mining

Abstract

The application of Data Mining (DM) techniques for DQ, often called Data Quality Mining (DQM), offers a wide range of possibilities for DQ assessment. The goal of this work is to propose a mechanism for data currency assessment using statistics and DM techniques. The proposed approach consists on estimating the validity period for the entities using a training set and then evaluating the probability of currency of the last known data value for each entity. The proposed scheme helps in two ways to lead to an always up-to-date database: it can warn if a certain data value is becoming obsolete, and it can inform the data manager about the best frequency for updating data.