New approach of recurrent neural network weight initialization

Abstract

This paper proposes a weight initialization strategy for a discrete-time recurrent neural network model. It is based on analyzing the recurrent network as a nonlinear system, and choosing its initial weights to put this system in the boundaries between different dynamics, i.e., its bifurcations. The relationship between the change in dynamics and training error evolution is studied. Two simple examples of the application of this strategy are shown: the identification of DC Induction motor and the detection of a physiological signal, a feature of a visual evoked potential brain signal. © Springer Science+Business Media B.V. 2009.