BI-POMDP" bounded, incremental partially-observable Markov-model planning

Abstract

Given the problem of planning actions for situations with uncertainty about the action outcomes, Markov models can effectively model this uncertainty and offer optimal actions. When the information about the world state is itself uncertain, partially observable Markov models are an appropriate extension to the basic Markov model. However, finding optimal actions for partially observable Markov models is a computationally difficult problem that in practice borders on intractability. Approximate or heuristic approaches, on the other hand, lose any guarantee of optimality or even any indication of how far from optimal they might be. In this paper, we present an incremental, search-based approximation for partially observable Markov models. The search is based on an incremental AND-OR search, using heuristic functions based on the underlying Markov model, which is more easily solved. In addition, the search provides a bound on the possible error of the approximation. We illustrate the method with results on problems taken from the related literature.