Usability of a co-designed eHealth prototype for caregivers: Combination study of three frameworks

Abstract

Background: Co-design (or the participation of users) has shown great potential in the eHealth domain, demonstrating positive results. Nevertheless, the co-design approach cannot guarantee the usability of the system designed, and usability assessment is a complex analysis to perform, as evaluation criteria will differ depending on the usability framework (or set of criteria) used. ISO (International Organization for Standardization) on usability (ISO 9241-210), Nielsen heuristic, and Garrett element of user experience inform different yet complementary aspects of usability. Objective: This study aims to assess the usability and user experience of a co-design prototype by combining 3 complementary frameworks. Methods: To help caregivers provide care for functionally impaired older people, an eHealth tool was co-designed with caregivers, health and social service professionals, and community workers assisting caregivers. The prototype was a website that aims to support the help-seeking process for caregivers (finding resources) and allow service providers to advertise their services (offering resources). We chose an exploratory study method to assess usability in terms of each objective. The first step was to assess users' first impressions of the website. The second was a task scenario with a think-aloud protocol. The final step was a semistructured interview. All steps were performed individually (with a moderator) in a single session. The data were analyzed using 3 frameworks. Results: A total of 10 participants were recruited, 5 for each objective of the website. We were able to identify several usability problems, most of which were located in the information design and interface design dimensions (Garrett framework). Problems in both dimensions were mainly coded as effectiveness and efficiency (ISO framework) and error prevention and match between the systemand the real world (Nielsen heuristic). Conclusions: Our study provided a novel contribution about usability analysis by combining the 3 different models to classify the problems found. This combination provided a holistic understanding of the usability improvements needed. It can also be used to analyze other eHealth products.