Learning the reward model of dialogue pomdps from data

Abstract

Spoken language communication between human and machines has become a challenge in research and technology. In particular, enabling the health care robots with spoken language interface is of great attention. Recently, there has been interest for modelling the dialogue manager of spoken dialogue systems using Partially Observable Markov Decision Processes (POMDPs). With the goal of modelling the dialogue manager of health care robots as dialogue POMDPs, we would like to learn the reward model of dialogue POMDPs from expert's data. In a previous paper work, we used an unsupervised learning method for learning the states, as well as the transition and observation functions of the dialogue POMDPs based on human human dialogues. Continuing our objective of learning the components of dialogue POMDPs from data, we introduce a novel inverse reinforcement learning algorithm for learning the reward function of the dialogue POMDP model. Based on the introduced method, and from an available corpus of data we construct a dialogue POMDP. Then, the learned dialogue policies, based on the learned POMDP, are evaluated. The empirical evaluation shows that the performance of the learned POMDP is higher than expert performance in non, low, and medium noise levels, but the high noise level. At the end, current limita- tions and future directions are addressed