Decomposing drama management in educational interactive narrative: A modular reinforcement learning approach

Abstract

Recent years have seen growing interest in data-driven approaches to personalized interactive narrative generation and drama management. Reinforcement learning (RL) shows particular promise for training policies to dynamically shape interactive narratives based on corpora of player-interaction data. An important open question is how to design reinforcement learning-based drama managers in order to make effective use of player interaction data, which is often expensive to gather and sparse relative to the vast state and action spaces required by drama management. We investigate an offline optimization framework for training modular reinforcement learning-based drama managers in an educational interactive narrative, CRYSTAL ISLAND. We leverage importance sampling to evaluate drama manager policies derived from different decompositional representations of the interactive narrative. Empirical results show significant improvements in drama manager quality from adopting an optimized modular RL decomposition compared to competing representations.