Better generalization with forecasts

Abstract

Predictive methods are becoming increasingly popular for representing world knowledge in autonomous agents. A recently introduced predictive method that shows particular promise is the General Value Function (GVF), which is more flexible than previous predictive methods and can more readily capture regularities in the agent's sensorimotor stream. The goal of the current paper is to investigate the ability of these GVFs (also called "forecasts") to capture such regularities. We generate focused sets of forecasts and measure their capacity for generalization. We then compare the results with a closely related predictive method (PSRs) already shown to have good generalization abilities. Our results indicate that forecasts provide a substantial improvement in generalization, producing features that lead to better value-function approximation (when computed with linear function approximators) than PSRs and better generalization to as-yet-unseen parts of the state space.