Weighted naïve Bayes classifiers by Renyi Entropy

Abstract

A weighted naïve Bayes classifier using Renyi entropy is proposed. Such a weighted naïve Bayes classifier has been studied so far, aiming at improving the prediction performance or at reducing the number of features. Among those studies, weighting with Shannon entropy has succeeded in improving the performance. However, the reasons of the success was not well revealed. In this paper, the original classifier is extended using Renyi entropy with parameter α. The classifier includes the regular naïve Bayes classifier in one end (α = 0.0) and naïve Bayes classifier weighted by the marginal Bayes errors in the other end (α = ∞). The optimal setting of α has been discussed analytically and experimentally. © Springer-Verlag 2013.