Envisioning Human-Machine Interaction in Future Warfare: Defence Industry Narratives on Human Control of Autonomous Weapon Systems

Abstract

The development of artificial intelligence and autonomous functions in the military domain has an immense impact on technologies being developed by the private defence industry. Defence firms contribute to the narratives and visions on autonomous weapon systems and the future of warfare, e.g. in the form of strategic marketing of their products. However, their role has so far been understudied, especially regarding autonomous weapon systems. As the normative debate revolves around aspects of human control, this work examines the narratives of (meaningful) human control in the marketing of autonomous military systems by defence manufacturers. Based on a comprehensive content analysis of twenty defence firms, we identified three main narratives, which envision autonomy as a military advantage, the role of the human in the future of warfare, and human-machine teaming. Based on the results, we argue that defence companies reproduce and adapt narratives which shape expectations and visions of human control of autonomous weapon systems in anticipation of emerging norms for (meaningful) human control. However, without specifications and verification mechanisms, there is no indication that human control will be meaningful.