A novel method for pruning decision trees

Abstract

Pruning decision trees is deemed an effective way of solving over-fitting in practice. Pruned decision trees usually have simpler structure and are expected to have higher generalization ability at the expense of classification accuracy. Nowadays, various pruning methods are available. However, the problem of how to make a trade-off between structural simplicity and classification accuracy has not been well solved. In this paper, we firstly propose a method to evaluate structural complexities of decision trees in pruning process. Based upon the method, we introduce a new measure for post-pruning decision trees, which takes into account both classification accuracy and structural complexity. The experimental results on 20 benchmark data sets from the UCI machine learning data repository show that the proposed method is competitively feasible for pruning decision trees. © 2009 IEEE.