Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems

Abstract

Objective: There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems. Materials and methods: Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method. Results: In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the "satisfaction" attribute (P =. 002). The number of problems identified using CW concerning the "learnability" attribute was significantly higher than those identified using HE (P =. 005). There was no significant difference between the number of problems identified by HE, based on different usability attributes (P =. 232). Results of CW showed a significant difference between the number of problems related to usability attributes (P