A new penalty-based criterion for model selection in regularized nonlinear models

Abstract

In this paper we describe a new penalty-based model selection criterion for nonlinear models which is based on the influence of the noise in the fitting. According to Occam's razor we should seek simpler models over complex ones and optimize the trade-off between model complexity and the accuracy of a model's description to the training data. An empirical derivation is developed and computer simulations for multilayer perceptron with weight decay regularization are made in order to show the efficiency and robustness of the method in comparison with other well-known criteria for nonlinear systems. © Springer-Verlag Berlin Heidelberg 2003.