Regularization by early stopping in single layer pereeptron training

Abstract

Adaptative training of the non-linear single-layer perceptron can lead to the Euclidean distance classifier and later to the standard Fisher linear discriminant function. On the way between these two classifiers one has a regularized discriminant analysis. That is equivalent to the "weight decay" regularization term added to the cost function. Thus early stopping plays a role of regularization of the network.