Structure optimization for echo state network based on contribution

Abstract

Echo State Network (ESN) is a recurrent neural network with a large, randomly generated recurrent part called the dynamic reservoir. Only the output weights are modified during training. However, proper balancing of the trade-off between the structure and performance for ESN remains a difficult task. In this paper, a structure optimized method for ESN based on contribution is proposed to simplify its network structure and improve its performance. First, we evaluate the contribution of reservoir neurons. Second, we present a pruning mechanism to remove the unimportant connection weights of reservoir neurons with low contribution. Finally, the new output weights are learned with the pseudo inverse method. The novel optimized ESN, named C-ESN, is tested on a Lorenz chaotic time-series prediction and an actual municipal sewage treatment system. The simulation results show that the C-ESN can have better prediction and generalization performance than ESN.