On the use of option policies for autonomous robot navigation

Abstract

We present here results and analysis on the use of fixed-duration option policies for navigation tasks in autonomous robotics. An option is sequence of actions taken by the robot without environmental feedback (open-loop control). Using options in replacement for actions leads to a more aggressive exploration of the state space, a convenient feature for tasks where autonomous learning of state trajectories is slow, such as mobile robot navigation. On the other hand, long sequences of actions taken in open loop can be dangerous, and from the point of view of learning can be counterproductive due to the exponential increase in the size of the policy space. We shown here that conservative options (corresponding to short sequences of actions) can be very effective, specially if their improved generalisation capabilities are combined with other mechanisms for increasing the generalisation efficiency of autonomous learning algorithms. © Springer-Verlag 2000.