Can User Reviews Indicate Usability Heuristic Issues?

Abstract

User reviews of mobile applications in online stores have been used as a source for numerous studies. Many focus on indicating new functional requirements and extracting problems regarding quality characteristics, such as Usability. One of the most known strategies to evaluate Usability is Nielsen's heuristics. We believe that user reviews in online stores can indicate heuristics issues, thus supporting evaluations more focused on the user feedback. Therefore, this study aims to investigate the presence of usability heuristic issues through user reviews. To achieve this goal, we collected and analyzed 200 reviews, split into 528 statements, from 10 free and paid Android and iOS apps. Our results showed that user reviews have certain potential to indicate heuristic issues, suggesting new and challenging future research for the CHI community.