Bayesian smoothing of decision tree soft predictions and evidential evaluation

Abstract

As for many classifiers, decision trees predictions are naturally probabilistic, with a frequentist probability distribution on labels associated to each leaf of the tree. Those probabilities have the major drawback of being potentially unreliable in the case where they have been estimated from a limited number of examples. Empirical Bayes methods enable the updating of observed probability distributions for which the parameters of the prior distribution are estimated from the data. This paper presents an approach of smoothing decision trees predictive binary probabilities with an empirical Bayes method. The update of probability distributions associated with tree leaves creates a correction concentrated on small-sized leaves, which improves the quality of probabilistic tree predictions. The amplitude of these corrections is used to generate predictive belief functions which are finally evaluated through the ensemblist extension of three evaluation indexes of predictive probabilities.