Incorporating learning into decision making in agent based models

Abstract

Most of the current work in social simulation is focused on building multi-agent systems that cooperate and collaborate together to exhibit some kind of a collective or social behavior. These agents are coded as rule-based or state-transition based systems. Often, the choice function of the agents is either hard-wired or dependent on environmental changes, but there is no explicit learning based on historical performance. There has been some recent work exploring methods for incorporating machine learning into multi-agent systems to capture adaptive behaviors. The goal of this paper is to expand upon the discussion around designing adaptive behaviors in agent based models by comparing multiple techniques for modeling learning, including: (1) applying machine learning and symbolic regression to use historical patterns to design learning mechanisms, (2) modeling behavioral economic principles to capture “irrational” or non-optimal learning, and (3) simulating reinforcement learning techniques such as q-learning to model a direct reward structure to improve learning outcomes at both the individual and group level. An example model has been built that applies these three learning techniques to simulate how people invest for retirement. Then, the outcomes of each learning technique simulated are used to identify lessons learned on when each technique should be applied.