Gaussian processes non-linear inverse reinforcement learning

Abstract

The authors analyse a Bayesian framework for posing and solving inverse reinforcement learning (IRL) problems that arise in decision-making and optimisation settings. The authors propose a non-parametric Bayesian model using Gaussian process (GP) and preference graphs, which offer an effective and computationally efficient method for ill-posed IRL problems in large or infinite state space. This approach only requires a finite number of demonstrations that is much less than the amount required for approximating the feature expectation or value functions in previous IRL methods. The proposed learning framework is expressive as it relies on a Bayesian structure in which assumptions are explicit and changeable. It is also robust in that it formalises on convex optimisation, which retains the promise of computationally manageable implementations for practical problems. To deal with more realistic IRL problems where the dynamics is also unknown, the GP model can be easily combined with the method to learn the dynamics at the same time. Experimental results prove the superiority of the authors method to current prevailing IRL algorithms with fewer numbers of demonstrations in both discrete and continuous state space.