Using J-pruning to reduce overfitting of classification rules in noisy domains

Abstract

The automatic induction of classification rules from examples is an important technique used in data mining. One of the problems encountered is the overfitting of rules to training data. This paper describes a means of reducing overfitting known as J-pruning, based on the J-measure, an information theoretic means of quantifying the information content of a rule, and examines its effectiveness in the presence of noisy data for two rule induction algorithms: one where the rules are generated via the intermediate representation of a decision tree and one where rules are generated directly from examples.