Episodic Retrieval for Model-Based Evaluation in Sequential Decision Tasks

Abstract

It has long been hypothesized that episodic memory supports adaptive decision making by enabling mental simulation of future events. Yet, attempts to characterize this process are surprisingly rare. On one hand, memory research is often carried out in settings that are far removed from ecological contexts of decision making. On the other hand, models of adaptive choice only invoke episodic memory in highly stylized terms, if at all. To address these gaps, we propose TCM-SR, a novel process-level model that grounds model-based evaluation in empirically informed dynamics of episodic recall. In this model, the probability of retrieving each available memory is governed by the successor representation, a biologically plausible world model in reinforcement learning. The evolution of these probabilities based on past retrievals, in turn, is dictated by the temporal context model, a prominent model of episodic retrieval. Through simulations and analytical derivations, we show that the patterns of episodic retrieval suggested by this model enables flexible computation of decision variables. On this basis, a number of previously described features of episodic memory might serve an adaptive purpose in sequential decision making. For instance, we show that the contiguity effect, a well-known bias in episodic retrieval, enables mental simulation via model-based rollouts to inform decisions. We also show that backward retrieval and emotional modulation improve generalization and the efficiency of decisions given limited experience. By bridging computational models across these two domains, we make several theoretical and empirical predictions linking episodic memory to adaptive choice in sequential tasks.