MML inference of oblique decision trees

Abstract

We propose a multivariate decision tree inference scheme by using the minimum message length (MML) principle (Wallace and Boulton, 1968; Wallace and Dowe, 1999). The scheme uses MML coding as an objective (goodness-of-fit) function on model selection and searches with a simple evolution strategy. We test our multivariate tree inference scheme on UCI machine learning repository data sets and compare with the decision tree programs C4.5 and C5. The preliminary results show that on average and on most data-sets, MML oblique trees clearly perform better than both C4.5 and C5 on both "right"/ "wrong" accuracy and probabilistic prediction - and with smaller trees, i.e., less leaf nodes. © Springer-Verlag Berlin Heidelberg 2004.