DRFLogitBoost: A double randomized decision forest incorporated with logitboosted decision stumps

Abstract

In this paper, a hybrid decision forest is constructed by double randomization of the original training set. In this decision forest, each individual base decision tree classifiers are incorporated with an additional classifier model, the Logitboosted decision stump. In the first randomization, the resamples to train the decision trees are extracted; in the second randomization, second set of resamples are generated from the out-of-bag samples of the first set of resamples. The boosted decision stumps are constructed on the second resamples. These extra resamples along with the resamples on which the base tree classifiers are trained, approximates the original training set. In this way we are utilizing the full training set to construct a hybrid decision forest with larger feature space. We have applied this hybrid decision forest in two real world applications; a) classifying credit scores, and b) short term extreme rainfall forecast. The performance of the hybrid decision forest in these two problems are compared with some well known machine learning methods. Overall results suggest that the new hybrid decision forest is capable of yielding commendable predictive performance. © 2012 Springer-Verlag.