Attractor Neural States: A Brain-Inspired Complementary Approach to Reinforcement Learning

Abstract

It is widely accepted that reinforcement learning (RL) mechanisms are optimal only if there is a predefined set of distinct states that are predictive of reward. This poses a cognitive challenge as to which events or combinations of events could potentially predict reward in a non-stationary environment. In addition, the computational discrepancy between two families of RL algorithms, model-free and model-based RL, creates a stability-plasticity dilemma, which in the case of interactive and competitive multiple brain systems poses a question of how to guide optimal decision-making control when there is competition between two systems implementing different types of RL methods. We argue that both computational and cognitive challenges can be met by infusing the RL framework as an algorithmic theory of human behavior with the strengths of the attractor framework at the level of neural implementation. Our position is supported by the hypothesis that ‘attractor states’ which are stable patterns of self-sustained and reverberating brain activity, are a manifestation of the collective dynamics of neuronal populations in the brain. Hence, when neuronal activity is described at an appropriate level of abstraction, simulations of spiking neuronal populations capture the collective dynamics of the network in response to recurrent interactions between these populations.