Don’t judge a book or health app by its cover: User ratings and downloads are not linked to quality

Abstract

Objective To analyse the relationship between health app quality with user ratings and the number of downloads of corresponding health apps. Materials and methods Utilising a dataset of 881 Android-based health apps, assessed via the 300-point objective Organisation for the Review of Care and Health Applications (ORCHA) assessment tool, we explored whether subjective user-level indicators of quality (user ratings and downloads) correlate with objective quality scores in the domains of user experience, data privacy and professional/clinical assurance. For this purpose, we applied spearman correlation and multiple linear regression models. Results For user experience, professional/clinical assurance and data privacy scores, all models had very low adjusted R squared values ( 0.05). For ORCHA scores multiple linear regression had adjusted R-squared = -.002. Conclusion This study highlights that widely available proxies which users may perceive to signify the quality of health apps, namely user ratings and downloads, are inaccurate predictors for estimating quality. This indicates the need for wider use of quality assurance methodologies which can accurately determine the quality, safety, and compliance of health apps. Findings suggest more should be done to enable users to recognise high-quality health apps, including digital health literacy training and the provision of nationally endorsed “libraries”.