MERCS: Multi-directional ensembles of regression and classification trees

Abstract

Learning a function fX→ Y that predicts Y from X is the archetypal Machine Learning (ML) problem. Typically, both sets of attributes (X, Y) have to be known before a model can be trained. When this is not the case, or when functions fX→Y are needed for varying X and Y, this may introduce significant overhead (separate learning runs for each function). In this paper, we explore the possibility of omitting the specification of X and Y at training time altogether, by learning a multi-directional, or versatile model, which will allow prediction of any Y from any X. Specifically, we introduce a decision tree-based paradigm that generalizes the well-known Random Forests approach to allow for multi-directionality. The result of these efforts is a novel method called MERCS: Multi-directional Ensembles of Regression and Classification treeS. Experiments show the viability of the approach.