Approximation of Bayesian discriminant function by neural networks in terms of Kullback-Leibler information

Abstract

Following general arguments on approximation Bayesian discriminant functions by neural networks, rigorously proved is that a three layered neural network, having rather a small number of hidden layer units, can approximate the Bayesian discriminant function for the two category classification if the log ratio of the a posteriori probability is a polynomial. The accuracy of approximation is measured by the Kullback- Leibler information. An extension to the multi-category case is also discussed.