A comparison of user testing and heuristic evaluation methods for identifying website usability problems

Abstract

This study compared the effectiveness and efficiency of two usability testing methods, user testing and heuristic evaluation. Thirty two participants took part in the study, sixteen for each of the two methods. Four measures were used to compare their performance: number of problems identified, severity of problems, type of problems and time taken to find problems. It was found that heuristic evaluation found nearly 5 times more individual problems than user testing, so could be seen as more effective. However, user testing found on average slightly more severe problems and took less time to complete than heuristic evaluation. Heuristic evaluation had a faster problem identification rate (number of seconds per problem found), so could also be seen as more efficient. While each method had advantages in the test both methods are seen as complementary to each other in practice.