A re-description based developmental approach to the generation of value functions for cognitive robots

Abstract

Motivation is a fundamental topic when implementing cognitive architectures aimed at lifelong open-ended learning in autonomous robots. In particular, it is of paramount importance for these types of architectures to be able to establish goals that provide purpose to the robot’s interaction with the world as well as to progressively learn value functions within its state space that allow reaching those goals whatever the starting point. This paper aims at exploring a developmental approach to the generation of high level neural network based value functions in complex continuous state spaces through a re-description process. This process starts by obtaining relatively simple Separable Utility Regions (SURs) which allow the system to consistently achieve goals, although not necessarily in the most efficient manner. The traces obtained by these SURs are then used to provide training data for a neural network based value function. Through a simple experiment with the Robobo robot, we show that this procedure can be more generalizable than attempting to directly obtain the value function through more traditional means.