Experience selection and problem choice in an exploratory learning system

Abstract

A fully autonomous exploratory learning system must perform two tasks that are not required of super- vised learning systems: experience selection and problem choice. Experience selection is the process of choosing informative training examples from the space of all possible examples. Problem choice is the process of identify- ing defects in the domain theory and determining which should be remedied next. These processes are closely related because the degree to which a specific experience is informative depends on the particular defects in the domain theory that the system is attempting to remedy. In this article we propose a general control structure for exploratory learning in which problem choice by an information-theoretic "curiosity" heuristic: the problem chosen then guides the selection of training examples. An implementation of an exploratory learning system based on this control structure is described, and a series of experimental results are presented.