Fast function approximation with hierarchial neural networks and their application to a reinforcement learning agent

Abstract

Current function approximators especially neural networks are often limited in several directions: most of the architectures can hardly be extended with more "informational" capacity, often neural networks with high capacity are too costly in calculation time (especially for an implementation on a microcontroller of a real world robot) and functions with high gradients can hardly be learned. The following approach shows, that these limitations can be overcome by using an adaptive hierarchical vector quantizing algorithm. With this algorithm the calculation time of a classification can decrease down to 0(log(n)). where n is the number of implemented prototypes. If a given number of prototypes can not carry the "information" of the function which has to be approximated, the "informational" capacity can be increased by adding prototypes. Proposed in this article the algorithm is tested in a Reinforcement Learning task. © Springer-Verlag Berlin Heidelberg 2001.