Ethics and Safety of Human-Machine Teaming

Abstract

With the advancement of autonomous systems, humans will be operating within close proximity to autonomous machines, which introduces safety considerations that extend beyond physical risks and harms. For human-machine teaming (HMT) operations, humans do not always play a hierarchical role, with autonomous systems often occupying the role of a teammate rather than a tool. The implementation of autonomous systems as teammates changes the safety landscape and has implications for ethics discussions around autonomous systems. The concept of safety for autonomous systems now encompasses additional considerations, including ethics. This paper advances the ethics discussions on autonomous systems by moving it from a view where humans function merely as onlookers in autonomous systems operations, to a view where the ethical considerations concern the nexus between humans and autonomous systems, captured in HMT with autonomous systems. How these considerations are captured in different safety management processes, such as risk assessments, remains an open question. This paper presents three key ethical principles for HMT; responsibility, traceability and trust, and provides considerations for capturing these principles in a risk assessment.