Evaluating Automated Vehicle Human–Machine Interfaces and the Relationship Between Preference and Performance Measures

Abstract

Driving automation systems have already entered the commercial market and further advancements will be introduced in the near future. Level 3 automated driving systems are expected to increase safety, comfort and traffic efficiency. For the human driver, these functions and according human-machine interfaces are a novel technology. In the human factors domain, research and development faces two challenges which are (1) the conceptualization of intuitive and easy to use interfaces and (2) the development of a methodological framework to evaluate these interfaces. In technology evaluation, a methodological phenomenon has frequently been reported which is called the preference-performance dissociation. It describes the outcome of studies where users’ preference (i.e., self-report) does not match their performance (i.e., interaction behavior). This phenomenon poses a threat to the evaluation of automated vehicle HMIs. Therefore, this chapter first reports a review and discussion of studies that investigated the operationalization of both measures. Moreover, the understanding (i.e., mental model) of automated vehicle HMIs is an influential precursor of performance and included in the present work. Using the insights of the operationalization part, the second part of this work aimed at finding out about factors that exert influence on preference and/or performance. Investigated factors were the number of use case repetitions, feedback on operator performance, user education and a post-hoc statistical analysis. To reach the operationalization and variation aims, three driving simulator studies with a total of N = 225 participants were conducted. The main outcomes were that (1) a sophisticated recommendation regarding preference questionnaire application is presented. Furthermore, (2) insights into the development of behavioral measures over time and their relation to a satisfaction measure could be given. Concerning mental models, (3) the present work showed that it takes repeated interaction to evolve accurately and gaze measures could also be used for capturing these processes. In addition, (4) feedback on operator performance was found to influence preference but not performance while (5) user education increased understanding and subsequent performance but did not affect preference. Eventually it showed that (6) users of different performance levels report similar preference. The theoretical contribution of this work lies in insights into the formation of the two sources of data and its potential to both explain and predict the preference-performance dissociation. The practical contribution lies in the recommendation for research methodology regarding how to operationalize measures and how to design user studies concerning the number of use cases and user education approaches. Finally, the results gained herein do not only apply to automated vehicle HMIs but might also be generalized to related domains such as aviation, robotics or health care.