A long-term memory enhanced echo state network and its optimization

Abstract

This paper proposes a new type of echo state network called Long-Term Memory Enhanced Echo State Network (LTME-ESN). By extending the ideas of the forget gate and the input gate from the LSTM, LTME-ESN modifies the equation for updating the state in a Leaky Integral Echo State Network (Leaky-ESN). By regulating the accumulation of information, the construction of extended periods of dependency between states may be delayed, allowing for adaptive management of the data transported from the previous state to the present one. In order to optimize parameters by using Stochastic Gradient Descent (SGD), this research presents a necessary condition for LTME-ESN to meet the properties of the echo state network. The study uses low-frequency sinusoidal, high-frequency sinusoidal, and chaotic time series to demonstrate the model's efficiency. Simulations show that LTME-ESN outperforms Leaky-ESN in prediction accuracy and variation.