Feasibility, acceptability and validity of SMS text messaging for measuring change in depression during a randomised controlled trial

Abstract

Background: Despite widespread popularity, text messaging has rarely been used for data collection in clinical research. This paper reports on the development, feasibility, acceptability, validity, and discriminant utility of a single item depression rating scale, delivered weekly via an automated SMS system, as part of a large randomised controlled trial. Methods: 755 depressed patients (BDI-II score ≥ 20) were recruited from primary care into a randomised trial of acupuncture versus counselling or usual care, and invited to opt into a repeated-measures text messaging sub-study. Two weeks following random allocation, trial participants were sent a weekly text message for 15 weeks. Texts were a single question asking, on a scale from 1 to 9, the extent to which they felt depressed. Feasibility and acceptability of the automated SMS system were evaluated according to cost, ease of implementation, proportion consenting, response rates, and qualitative feedback. Concurrent validity was estimated by correlating SMS responses with the Patient Health Questionnaire (PHQ-9). SMS responses were compared between groups over time to explore treatment effects. Results: 527 (69.8%) trial participants consented to the texting sub-study, of whom 498 (94.5%) responded to at least one message. Participants provided a valid response to an average of 12.5 messages. Invalid responses accounted for 1.1% of texts. The automated SMS system was quick to set-up, inexpensive, and well received. Comparison of PHQ-9 and SMS responses at 3 months demonstrated a moderate to high degree of agreement (Kendall's tau-b = 0.57, p < 0.0001, n = 220). SMS depression scores over the 15 weeks differed significantly between trial arms (p = 0.007), with participants allocated to the acupuncture and counselling arms reporting improved depression outcomes compared to usual GP care alone, which reached statistical significance ten weeks after randomisation. Overall, the single item SMS scale also appeared more responsive to changes in depression, resulting from treatment, than the PHQ-9. Conclusions: Automated SMS systems offer a feasible and acceptable means of monitoring depression within clinical research. This study provides clear evidence to support the regular use of a simple SMS scale as a sensitive and valid outcome measure of depression within future randomised controlled trials. Trial registration: Current Controlled Trials - ISRCTN63787732