Trust Issues with Trust Scales: Examining the Psychometric Quality of Trust Measures in the Context of AI

Abstract

Trust is crucial for human interaction with artificial intelligence (AI) and is frequently measured through questionnaires or rating scales. One commonly used questionnaire in AI research is the Trust between People and Automation scale (TPA). However, its psychometric quality has yet to be examined in the context of AI. More recently, a Trust Scale for Explainable AI (TXAI) was recommended but not empirically evaluated. In this study, we assessed the psychometric qualities of both scales, using confirmatory and exploratory factor analyses to test the scales' validity and coefficients α and ω for reliability estimation. Our results suggested good psychometric quality for the TXAI after removing one item. Concerning the TPA, acceptable quality was only achieved when using a two-factor model (trust and distrust) and after removing two items. We provide recommendations for using the two scales and evidence to distinguish trust and distrust as separate psychological constructs.