Fluctuation-learning relationship in recurrent neural networks

Abstract

Learning speed depends on both task structure and neural dynamics prior to learning, yet a theory connecting them has been missing. Inspired by the fluctuation-response relation, we derive two formulae linking neural dynamics to learning. Initial learning speed is proportional to the covariance between pre-learning spontaneous activity and network’s input-evoked response, independent of the learning rule. For Hebb-type learning, initial speed scales with the variance of activity along target and input directions. These results apply across tasks including input-output mapping and time-series generation. Numerical simulations across diverse models validate the formulae beyond the theoretical-derivation’s assumptions. Although derived for early learning, the formulae predict total learning time. A straightforward implication is learning is faster when task-relevant directions align with high-variance spontaneous activities, consistent with empirical findings. Our framework establishes how the geometrical relationship between pre-learning dynamics and task directions governs learning speed, independent of details of tasks.