Measure for Measure: How Do We Assess Human Autonomy Teaming?

Abstract

The increasing use of advanced automation and intelligent support systems provides an opportunity for increased efficiency and minimizing the requirement of presenting the human with laborious, dangerous and repetitive tasks. It is important to remember that although a large degree of this capability is technology-focused, the significance of the human must not be relegated or dismissed. The human component is tightly coupled to the whole system and should be viewed as working in harmony with the autonomous system. In many instances we are seeing systems that bring both human and autonomy closer together, working as teams. With both the human and autonomy working together it is important to be able to evaluate the effectiveness of this teaming arrangement. However, methods for measuring the effectiveness of human-autonomy teaming tends to focus solely on the task, with little consideration being given to how the human integrates within the system. This paper explores the factors associated with task-based measures and highlights key cognitive factors that should be considered. It is unlikely that a single metric can be used to provide a score, but rather a combination of metrics that pull in different components of the human-autonomy team. The ability to measure the effectiveness of human-autonomy teams will not only provide feedback as to current manifestations of teaming, but has implications into their future design.