Isolating red flags to enhance diagnosis (I-RED): An experimental vignette study

Abstract

Objective: To investigate effects of a cognitive intervention based on isolation of red flags (I-RED) on diagnostic accuracy of 'do-not-miss diagnoses.' Design: A 2 × 2 randomized case vignette-based experiment with manipulation of I-RED strategy between subjects and case complexity within subjects. Setting: Two university-based residency programs. Participants: One-hundred and nine pediatric residents from all levels of training. Interventions: Participants were randomly assigned to the I-RED vs. control group, and within each group, they were further randomized to the order in which they saw simple and complex cases. The I-RED strategy involved an instruction to look for a constellation of symptoms, signs, clinical data or circumstances that should heighten suspicion for a serious condition. Main Outcome Measures: Primary outcome was diagnostic accuracy, scored as 1 if any of the three differentials given by participants included the correct diagnosis, and 0 if not. We analyzed effects of I-RED strategy on diagnostic accuracy using logistic regression. Results: I-RED strategy did not yield statistically higher diagnostic accuracy compared to controls (62 vs. 48%, respectively; odd ratio = 2.07 [95% confidence interval, 0.78-5.5], P = 0.14) although participants reported higher decision confidence compared to controls (7.00 vs. 5.77 on a scale of 1 to 10, P < 0.02) in simple but not complex cases. I-RED strategy significantly shortened time to decision (460 vs. 657 s, P < 0.001) and increased the number of red flags generated (3.04 vs. 2.09, P < 0.001). Conclusions: A cognitive strategy of prompting red flag isolation prior to differential diagnosis did not improve diagnostic accuracy of 'do-not-miss diagnoses.' Given the paucity of evidence-based solutions to reduce diagnostic error and the intervention's potential effect on confidence, findings warrant additional exploration.