A dynamic allocation method of basis functions in reinforcement learning

Abstract

In this paper, we propose a dynamic allocation method of basis functions, an Allocation/Elimination Gaussian Softmax Basis Function Network (AE-GSBFN), that is used in reinforcement learning. AE-GSBFN is a kind of actor-critic method that uses basis functions. This method can treat continuous high-dimensional state spaces, because basis functions required only for learning are dynamically allocated, and if an allocated basis function is identified as redundant, the function is eliminated. This method overcomes the curse of dimensionality and avoids a fall into local minima through the allocation and elimination processes. To confirm the effectiveness of our method, we used a maze task to compare our method with an existing method, which has only an allocation process. Moreover, as learning of continuous high-dimensional state spaces, our method was applied to motion control of a humanoid robot. We demonstrate that the AE-GSBFN is capable of providing better performance than the existing method. © Springer-Verlag Berlin Heidelberg 2004.