Multivariate Predictive Clustering Trees for Classification

Abstract

Decision trees are well established machine learning models that combined in ensembles produce state-of-the-art predictive performance. Predictive clustering trees are a generalization of standard classification and regression trees towards structured output prediction and semi-supervised learning. Most of the research attention is on univariate decision trees, whereas multivariate decision trees, in which multiple attributes can appear in a test, are less widely used. In this paper, we present a multivariate variant of predictive clustering trees, and experimentally evaluate it on 12 classification tasks. Our method shows good predictive performance and computational efficiency, and we illustrate its potential for performing feature ranking.