Mining Dynamics: Using Data Mining Techniques to Analyze Multi-agent Learning

Abstract

Analyzing the learning dynamics in multi-agent systems (MASs) has received growing attention in recent years. Theoretical analysis of the dynamics was only possible in simple domains and simple algorithms. When one or more of these restrictions do not apply, theoretical analysis becomes prohibitively difficult, and researchers rely on experimental analysis instead. In experimental analysis, researchers have used some global performance metric(s) as a rough approximation to the internal dynamics of the adaptive MAS. For example, if the overall payoff improved over time and eventually appeared to stabilize, then the learning dynamics were assumed to be stable as well. In this paper, we promote a middle ground between the thorough theoretical analysis and the high-level experimental analysis. We introduce the concept of mining dynamics and propose data-mining-based methodologies to analyze multi-agent learning dynamics. Using our methodologies, researchers can identify clusters of learning parameter values that lead to similar performance, and discover frequent sequences in agent dynamics. We verify the potential of our approach using the well-known iterated prisoner's dilemma (with multiple states) domain.