Learning autonomy in two or three steps: linking open-ended development, authority, and agency to motivation

Abstract

In this paper we connect open-ended development, authority, agency, and motivation through (1) an analysis of the demands of existing in a complex world and (2) environmental appraisal in terms of affordance content and the complexity to select appropriate behavior. We do this by identifying a coherent core from a wide range of contributing fields. Open-ended development is a structured three-step process in which the agent first learns to master the body and then aims to make the mind into a reliable tool. Preconditioned on success in step two, step three aims to effectively co-create an optimal living environment. We argue that these steps correspond to right-left-right hemispheric dominance, where the left hemisphere specializes in control and the right hemisphere in exploration. Control (e.g., problem solving) requires a closed and stable world that must be maintained by external authorities or, in step three, by the right hemisphere acting as internal authority. The three-step progression therefore corresponds to increasing autonomy and agency. Depending on how we appraise the environment, we formulate four qualitatively different motivational states: submission, control, exploration, and consolidation. Each of these four motivational states has associated reward signals of which the last three-successful control, discovery of novelty, and establishing new relations-form an open-ended development loop that, the more it is executed, helps the agent to become progressively more agentic and more able to co-create a pleasant-to-live-in world. We conclude that for autonomy to arise, the agent must exist in a (broad) transition region between order and disorder in which both danger and opportunity (and with that open-ended development and motivation) are defined. We conclude that a research agenda for artificial cognitive system research should include open-ended development through intrinsic motivations and ascribing more prominence to right hemispheric strengths.