Multivariate decision tree function approximation for reinforcement learning

Abstract

In reinforcement learning, when dimensionality of the state space increases, making use of state abstraction seems inevitable. Among the methods proposed to solve this problem, decision tree based methods could be useful as they provide automatic state abstraction. But existing methods use univariate, therefore axis-aligned, splits in decision nodes, imposing hyper-rectangular partitioning of the state space. In some applications, multivariate splits can generate smaller and more accurate trees. In this paper, we use oblique decision trees as an instance of multivariate trees to implement state abstraction for reinforcement learning agents. Simulation results on mountain car and puddle world tasks show significant improvement in the average received rewards, average number of steps to finish the task, and size of the trees both in learning and test phases. © 2010 Springer-Verlag.