Emergence of a resonance in machine learning

Abstract

The benefits of noise to applications of nonlinear dynamical systems through mechanisms such as stochastic and coherence resonances have been well documented. Recent years have witnessed a growth of research in exploiting machine learning to predict nonlinear dynamical systems. It has been known that noise can act as a regularizer to improve the training performance of machine learning. Utilizing reservoir computing as a paradigm, we find that injecting noise to the training data can induce a resonance phenomenon with significant benefits to both short-term prediction of the state variables and long-term prediction of the attractor. The optimal noise level leading to the best performance in terms of the prediction accuracy, stability, and horizon can be identified by treating the noise amplitude as one of the hyperparameters for optimization. The resonance phenomenon is demonstrated using two prototypical high-dimensional chaotic systems.