Gödel agents in a scalable synchronous agent framework

Abstract

A synchronous framework for the interaction of an agent and an environment based on Moore machines is introduced. Within this framework, the notion of a Gödel agent is defined relative to a family of agents and environments and a time horizon T. A Gödel agent is the most flexible, adapting and self-improving agent with regard to the given environment family. It scores well across many environments, and not only in a selected few. Ideas from infinite game theory and ruin theory are used to get well-defined limits for T→∞ by introducing negative goals or repellors. This allows to score actions of the agent by how probable an action makes the survival of the agent till the end of time. Score functions of this type will be called “liveness” scores, and they provide a solution to the horizon problem from a foundational point of view. Additionally, by varying the agent and environment families, one gets a scalable and flexible testbed which could prove to be well-suited for analyzing phenomena of adaptation and self-improvement, both theoretically and empirically.