A phenomenologically justifiable simulation of mental modeling

Abstract

Real-world agents need to learn how to react to their environment. To achieve this, it is crucial that they have a model of this environment that is adapted during interaction and although important aspects may be hidden. This paper presents a new type of model for partially observable environments that enables an agent to represent hidden states but can still be generated and queried in realtime. Agents can use such a model to predict the outcomes of their actions and to infer action policies. These policies turn out to be better than the optimal policy in a partially observable Markov decision process as it can be inferred, for example, by Q- or Sarsa-learning. The structure and generation of these models are motivated both by phenomenological considerations from semiotics and the philosophy of mind. The performance of these models is compared to a baseline of Markov models for prediction and interaction in partially observable environments.