Expected usability is not a valid indicator of experienced usability

Abstract

Usability is a core construct of website evaluation and inherently defined as interactive. Yet, when analysing first impressions of websites, expected usability, i.e., before use, is of interest. Here we investigate to what extend ratings of expected usability are related to (a) experienced usability, i.e., ratings after use, and (b) objective usability measures, i.e., task performance. Furthermore, we try to elucidate how ratings of expected usability are correlated to aesthetic judgments. In an experiment, 57 participants submitted expected usability ratings after the presentation of website screenshots in three viewing-time conditions (50, 500, and 10,000 ms) and after an interactive task (experienced usability). Additionally, objective usability measures (task completion and duration) and subjective aesthetics evaluations were recorded for each website. The results at both the group and individual level show that expected usability ratings are not significantly related either to experienced usability or objective usability measures. Instead, they are highly correlated with aesthetics ratings. Taken together, our results highlight the need for interaction in empirical website usability testing, even when exploring very early usability impressions. In our study, user ratings of expected usability were no valid proxy neither for objective usability nor for experienced website usability.