InfoMax Control for Acoustic Exploration of Objects by a Mobile Robot
Abstract
Recently, information gain has been proposed as a candidate intrinsic motivation for lifelong learning agents that may not always have a specific task. In the InfoMax control frame- work, reinforcement learning is used to find a control policy for a POMDP in which movement and sensing actions are se- lected to reduce Shannon entropy as quickly as possible. In this study, we implement InfoMax control on a robot which can move between objects and perform sound-producing ma- nipulations on them. We formulate a novel latent variable mixture model for acoustic similarities and learn InfoMax po- lices that allow the robot to rapidly reduce uncertainty about the categories of the objects in a room.We find that InfoMax with our improved acoustic model leads to policies which lead to high classification accuracy. Interestingly, we also find that with an insufficient model, the InfoMax policy eventually learns to bury its head in the sand to avoid getting addi- tional evidence that might increase uncertainty. We discuss the implications of this finding for InfoMax as a principle of intrinsic motivation in lifelong learning agents.
Sign up today - FREE
Mendeley saves you time finding and organizing research. Learn more
- All your research in one place
- Add and import papers easily
- Access it anywhere, anytime