Unpacking the “Black box”: How an SMS-based continuing medical education intervention improved medical knowledge among HIV clinicians in Vietnam

Abstract

Background: A mobile-based continuing medical education (mCME) intervention implemented over 6 months between 2016 and 2017, consisting of daily SMS multiple choice quizzes and access to online daily readings and CME courses, was shown to be effective in increasing medical knowledge among HIV providers in Vietnam. We hypothesized this improvement was a result of “lateral learning,” a process in which the daily SMS quizzes acted as a stimulus for interacting with other study materials. Methods: We explored how study materials directly provided by the intervention—the daily readings and the online CME courses—and independent study behaviors, such as using medical textbooks and reviewing national guidelines, contributed to medical knowledge as measured by baseline and endline exams. At baseline, there were 53 participants each in the intervention and control groups (N=106). Using linear regression models, we estimated the association between intervention-prompted and independent study behaviors and endline test scores. We also conducted a series of interaction analyses to test the extent to which the effect of daily quiz performance on endline test scores depended on use of the intervention-prompted or independent study materials. Finally, we estimated the proportion of variance in endline test scores explained by each of the intervention-prompted behaviors. Results: The average medical knowledge test score among all participants was 46% at baseline and 54% at endline. Among the intervention group, 82% of the daily quizzes were answered, although only about half were answered correctly. Responding to the daily quizzes (ß=0.24; P=.05), quiz performance (ß=0.42; P