Active inference, preference learning and adaptive behaviour

Abstract

The ability to adapt to a changing environment underwrites sentient behaviour e.g., wearing a raincoat when walking in the rain but removing it when indoors. In such instances, agents act to satisfy some preferred mode of behaviour that leads to predictable states necessary for survival, i.e., states that are characteristic of that agent. In this chapter, we describe how active inference agents, equipped with preference learning, can exhibit these distinct behavioural modes – influenced by environment dynamics – to aptly trade-off between preference satisfaction and exploration. We validate this in a modified OpenAI Gym FrozenLake environment (without any extrinsic signal) with and without volatility under a fixed model of the environment. In a static (i.e., without volatility) environment, preference-learning agents accumulate confident (Bayesian) beliefs about their behaviour and act to satisfy them. In contrast, volatile dynamics led to preference uncertainty and exploratory behaviour. This demonstrates that active inference agents, equipped with preference learning, have the appropriate machinery to (i) engage in adaptive behaviour under appropriate levels of volatility, and (ii) learn context-dependent subjective preferences.