Understanding the spirit of a norm: Challenges for norm-learning agents

Abstract

Social and moral norms are a fabric for holding human societies together and helping them to function. As such they will also become a means of evaluating the performance of future human–machine systems. While machine ethics has offered various approaches to endowing machines with normative competence, from the more logic-based to the more data-based, none of the proposals so far have considered the challenge of capturing the “spirit of a norm,” which often eludes rigid interpretation and complicates doing the right thing. We present some paradigmatic scenarios across contexts to illustrate why the spirit of a norm can be critical to make explicit and why it exposes the inadequacies of mere data-driven “value alignment” techniques such as reinforcement learning RL for interactive, real-time human–robot interaction. Instead, we argue that norm learning, in particular, learning to capture the spirit of a norm, requires combining common-sense inference-based and data-driven approaches.