Prioritizing user concerns in app reviews – A study of requests for new features, enhancements and bug fixes

Abstract

Context:: App developers spend exhaustive manual efforts towards the identification and prioritization of informative end-user reviews. Informative reviews are those that express end-users’ requests for new features, bug fixes and possible enhancements. Problem Statement: While prior studies have proposed approaches to convert app reviews into actionable knowledge, these are limited in utility due to being domain knowledge dependent or manually-based. Objective:: In this study, in order to facilitate app maintenance and evolution cycles, we develop two novel automated prioritization techniques to rank informative reviews, and also compare their performances. Method:: We developed the techniques comprising of entropy, frequency, TF-IDF and sentiment scoring methods using reviews from four popular apps comprising more than 1000 informative reviews in each app. Time and accuracy metrics were then used to measure the performance of our techniques. We performed evaluations where the ranking outcomes generated by our techniques were compared to those provided by regular app users and developers using two rounds of evaluations (internal and external evaluations). Results:: Our results show that the time required for prioritization was in the range of 17–25 s and the accuracy of prioritization was in the range of 73–90%. Conclusion:: These are promising outcomes when compared to prior work, where our outcomes were 4% and 185% better in terms of accuracy and time respectively. Thus, it is anticipated that our proposed techniques could support app developers in identifying and prioritizing informative reviews.