How Delays Affect Neural Dynamics and Learning

Abstract

We investigate the effects of delays on the dynamics and, in particular, on the oscillatory properties of simple neural network models. We extend previously known results regarding the effects of delays on stability and convergence properties. We treat in detail the case of ring networks for which we derive simple conditions for oscillating behavior and several formulas to predict the regions of bifurcation, the periods of the limit cycles and the phases of the different neurons. These results in turn can readily be applied to more complex and more biologically motivated architectures, such as layered networks. In general, the main result is that delays tend to increase the period of oscillations and broaden the spectrum of possible frequencies, in a quantifiable way. Simulations show that the theoretically predicted values are in excellent agreement with the numerically observed behavior. Adaptable delays are then proposed as one additional mechanism through which neural systems could tailor their own dynamics. Accordingly, we derive recurrent backpropagation learning formulas for the adjustment of delays and other parameters in networks with delayed interactions and discuss some possible applications. © 1994 IEEE