Density- and complexity-regularization gaussian mixture bayesian classifier

Abstract

We regularize Gaussian mixture Bayesian (GMB) classifier in terms of the following two points: |1) class-conditional probability density functions, and 2) complexity as a classifier. For the former, we employ the Bayesian regularization method proposed by Ormoneit and Tresp, which is derived from the maximum a posteriori (MAP) estimation framework. For the latter, we use a discriminative MDL-based model selection method proposed by us. In this paper, we optimize the hyper parameters in 1) and 2) simultaneously with respect to the discriminative MDL criterion, aiming to auto-configure the hyperparameter setting for the best classification performance. We show the effectiveness of the proposed method through some experiments on real datasets.