A representation theorem for decisions about causal models

Abstract

Given the likely large impact of artificial general intelligence, a formal theory of intelligence is desirable. To further this research program, we present a representation theorem governing the integration of causal models with decision theory. This theorem puts formal bounds on the applicability of the submodel hypothesis, a normative theory of decision counterfactuals that has previously been argued on a priori and practical grounds, as well as by comparison to theories of counterfactual cognition in humans. We are able to prove four conditions under which the submodel hypothesis holds, forcing any preference between acts to be consistent with some utility function over causal submodels. © 2012 Springer-Verlag.