Understanding Perceived Quality of Mobile Apps Using DistilBERT Model

Abstract

Measuring the perceived quality of mobile applications is a pivotal consideration for app developers, users, and platforms, as perceived quality can be measured by getting user reviews. Yet, the common sentiment analysis (SA) does not focus on more linguistic indicators of app quality. In this study, a model based on DistilBERT is suggested to predict the perceived app quality and classify end-user reviews on a high-quality or low-quality scale. Manually annotated data was applied, using quality parameters (informativeness, clarity, relevance, specificity, coherence, level of details, and actionable usability feedback), while also integrating appraisal theory’s key components, i.e., affect, appreciation, and judgment. Affect is related to the emotional tendency of the user. Appreciation concerns functional effectiveness, and the role of judgment is related to evaluating performance. These linguistic indicators offer deeper insights into how users emotionally and cognitively assess app quality. In order to certify the interpretability of the model, an Explainable AI (XAI) solution, namely SHAP (SHapley Additive exPlanations), was utilised to cross-check the annotations and confirm the emphasis of the model on primary measures of quality, such as usability, stability, and performance. The 5-fold stratified cross-validation measurement updated this model, yielding high performance with an average accuracy of 92.88%. It was evaluated based on reviews of seven trending mobile apps, including ESPN, Snapchat, Skype, Opera Mini, MX Player, VLC Media Player, and Daraz Online Shopping, and performed better than the baseline strategies. The findings confirm the strength and performance of the model, which presents a scalable automated app quality evaluation framework.