Positive bias in the ‘Trust in Automated Systems Survey’? An examination of the Jian et al. (2000) scale

Abstract

Measuring trust in technology is a mainstay in Human Factors research. While trust may not perfectly predict reliance on technology or compliance with alarm signals, it is routinely used as a design consideration and assessment goalpost. Several methods of measuring trust have been employed in the past decades, but one self-report measure stands out due to its popular use, the Trust in Automated Systems Survey (Jian, Bisantz, & Drury, 2000). We conducted a study to assess whether the survey could create biased responses, and found evidence the original scale is in fact skewed toward positive ratings. Assessing the literature revealed the survey has been used in unaltered form across at least 100 different reports and remains frequently administered - therefore, the potential impact of this bias may be widespread. Future directions, considerations, and caveats for our assessment, and for using this scale, are discussed.