Mobile app validation: a digital health scorecard approach

Abstract

While digital health solutions continue to grow in number and in complexity, the ability for stakeholders in healthcare to easily discern quality lags far behind. This challenge is in part due to the lack of a transparent and standardized approach to validation. Evaluation of mobile health applications (apps) is further burdened by low barriers to development and direct-to-user marketing, leading to a crowded and confusing landscape. In this context, we investigated the pragmatic application of a previously described framework for digital health validation, the Digital Health Scorecard, in a cohort of 22 popular mobile health oncology apps. The apps evaluated using this framework performed poorly, scoring 49.4% across all evaluation criteria as a group. Performance across component domains varied considerably with cost scoring highest at 100%, usability at 56.7%, technical at 37.3%, and clinical at 15.9%. satisfaction of prospectively determined end-user requirements derived from patient, family, and clinician consensus scored 37.2%. While cost outperformed consistently and usability was adequate, the results also suggested that apps suffered from significant technical limitations, were of limited clinical value, and generally did not do what end users wanted. These large gaps further support the need for transparent and standardized evaluation to help all stakeholders in healthcare improve the quality of mobile health.