Reconciling trust and control in the military use of artificial intelligence

Abstract

In regulating military applications of artificial intelligence (AI), the relationship between humans and the AI systems they operate is of central importance. AI developers commonly frame the desired human-AI relationship in terms of 'trust', aiming to make AI systems sufficiently 'trustworthy' for the task at hand and foster appropriate levels of human 'trust' in complex, often inscrutable, AI systems. Meanwhile, in legal and ethical discussions, the challenge is generally framed as ensuring that humans retain 'control' over AI such that responsible operators can reliably guide the behaviour of AI systems as required by legal and other norms. Surprisingly, few have asked whether the paradigms of 'trust' and 'control' are guiding development of the human-AI relationship in the same direction. This paper outlines the nature of trust and control as they relate to regulation of the human-AI relationship and surveys some challenges which arise in regulating the military uptake of AI systems.