Adaptive natural language generation in dialogue using reinforcement learning

Abstract

This paper presents a new model for adap-tive Natural Language Generation (NLG) in dialogue, showing how NLG problems can be approached as statistical planning problems using Reinforcement Learning. This approach brings a number of theo-retical and practical benefits such as fine-grained adaptation, generalization, and au-tomatic (global) optimization. We present the model and related work in statisti-cal/trainable NLG, discuss its applications, and provide a demonstration of the ap-proach, showing policy learning for adaptive information presentation decisions (Con-trast, Cluster, or List items). An adap-tive NLG policy learned in our framework shows a statistically significant 27% relative increase in reward over an " RL-majority " baseline policy for the same task. We thereby also show that that such NLG prob-lems should be approached in combination with dialogue management decisions, and we show how to jointly optimize NLG and dialogue management plans.