POMDP filter: Pruning POMDP value functions with the Kaczmarz iterative method

Abstract

In recent years, there has been significant interest in developing techniques for finding policies for Partially Observable Markov Decision Problems (POMDPs). This paper introduces a new POMDP filtering technique that is based on Incremental Pruning [1], but relies on geometries of hyperplane arrangements to compute for optimal policy. This new approach applies notions of linear algebra to transform hyperplanes and treat their intersections as witness points [5]. The main idea behind this technique is that a vector that has the highest value at any of the intersection points must be part of the policy. IPBS is an alternative of using linear programming (LP), which requires powerful and expensive libraries, and which is subjected to numerical instability. © 2010 Springer-Verlag.