Revisiting “Recurrent World Models Facilitate Policy Evolution”

Abstract

This paper contributes a detailed analysis of the architecture of Ha and Schmidhuber [5]. The original paper proposes an architecture comprising 3 main components: a “visual” module, a “memory” module; and a controller. As a whole, such architecture performed well in challenging domains. We investigate how each of the aforementioned components contributes individually to the final performance of the system. Our results shed additional light on the role of the different components in the overall behavior of the agent, and illustrate how the different design options affect the behavior of the resulting agent.