Usability of the Experience Sampling Method in Specialized Mental Health Care: Pilot Evaluation Study

Abstract

Background: Mental health problems occur in interactions in daily life. Yet, it is challenging to bring contextual information into the therapy room. The experience sampling method (ESM) may facilitate this by assessing clients’ thoughts, feelings, symptoms, and behaviors as they are experienced in everyday life. However, the ESM is still primarily used in research settings, with little uptake in clinical practice. One aspect that may facilitate clinical implementation concerns the use of ESM protocols, which involves providing practitioners with ready-to-use ESM questionnaires, sampling schemes, visualizations, and training. Objective: This pilot study’s objective was to evaluate the usability of an ESM protocol for using the ESM in a specialized mental health care setting. Methods: We created the ESM protocol using the m-Path software platform and tested its usability in clinical practice. The ESM protocol consists of a dashboard for practitioners (ie, including the setup of the template and data visualizations) and an app for clients (ie, for completing the ESM questionnaires). A total of 8 practitioners and 17 clients used the ESM in practice between December 1, 2020, and July 31, 2021. Usability was assessed using questionnaires, ESM compliance rates, and semistructured interviews. Results: The usability was overall rated reasonable to good by practitioners (mean scores of usability items ranging from 5.33, SD 0.91, to 6.06, SD 0.73, on a scale ranging from 1 to 7). However, practitioners expressed difficulty in personalizing the template and reported insufficient guidelines on how to use the ESM in clinical practice. On average, clients completed 55% (SD 25%) of the ESM questionnaires. They rated the usability as reasonable to good, but their scores were slightly lower and more variable than those of the practitioners (mean scores of usability items ranging from 4.18, SD 1.70, to 5.94, SD 1.50 on a scale ranging from 1 to 7). Clients also voiced several concerns over the piloted ESM template, with some indicating no interest in the continued use of the ESM. Conclusions: The findings suggest that using an ESM protocol may facilitate the implementation of the ESM as a mobile health assessment tool in psychiatry. However, additional adaptions should be made before further implementation. Adaptions include providing training on personalizing questionnaires, adding additional sampling scheme formats as well as an open-text field, and creating a dynamic data visualization interface. Future studies should also identify factors determining the suitability of the ESM for specific treatment goals among different client populations.