Maximum of marginal likelihood criterion instead of cross-validation for designing of artificial neural networks

Abstract

The cross-validation method is commonly applied in the design of Artificial Neural Networks (ANNs). In the paper the design of ANN is related to searching for an optimal value of the regularization coefficient or the number of neurons in the hidden layer of network. Instead of the cross-validation procedure, the Maximum of Marginal Likelihood (MML) criterion, taken from Bayesian approach, can be used. The MML criterion, applied to searching for the optimal values of design parameters of neural networks, is illustrated on two examples. The obtained results enable us to formulate conclusions that the MML criterion can be used instead of the cross-validation method (especially for small data sets), since it permits the design of ANNs without formulation of a validation set of patterns. © 2008 Springer-Verlag Berlin Heidelberg.