User Simulation in the Development of Statistical Spoken Dialogue Systems

Abstract

Statistical approaches to dialogue management have steadily increased in popularity over the last decade. Recent evaluations of such dialogue managers have shown their feasibility for sizeable domains and their advantage in terms of increased robustness. Moreover, simulated users have shown to be highly beneficial in the development and testing of dialogue managers and in particular, for training statistical dialogue managers. Learning the optimal policy of a POMDP dialogue manager is typically done using the reinforcement learning (RL), but with the RL algorithms that are commonly used today, this process still relies on the use of a simulated user. Data-driven approaches to user simulation have been developed to train dialogue managers on more realistic user behaviour. This chapter provides an overview of user simulation techniques and evaluation methodologies. In particular, recent developments in agenda-based user simulation, dynamic Bayesian network-based simulations and inverse reinforcement learning-based user simulations are discussed in detail. Finally, we will discuss ongoing work and future challenges for user simulation.