Localized bayes estimation for non-identifiable models

Abstract

Hierarchical learning machines such as neural networks are now being used in many applications. Although the Bayes ensemble learning gives the good generalization performance in such hierarchical learning machines, it is difficult to realize the posterior distribution because of the singularities in the parameter space. In this paper, we propose a new learning algorithm which enables us to construct a localized posterior distribution. We call this method Localized Bayes estimation and theoretically show that it attains the smaller generalization error in reduced rank approximations. © Springer-Verlag Berlin Heidelberg 2006.