Effects of deliberate reflection on diagnostic accuracy, confidence and diagnostic calibration in dermatology

Abstract

Background: Deliberate reflection on initial diagnoses improved diagnostic accuracy in internal medicine and general practice, but it is unknown if the same occurs in specialties that rely mostly on visual perception, such as dermatology. Moreover, whether reflection influences diagnostic calibration has not been studied yet. Diagnostic calibration, the relationship between diagnostic accuracy and confidence in that accuracy, affects diagnostic performance because overconfidence tends to induce premature closure. This study evaluated the effects of deliberate reflection on diagnostic accuracy and diagnostic calibration in dermatology. Methods: Sixty-one sixth-year students from a Brazilian medical school were allocated to either a reflection group (RG) or a control group (CG). In both groups, students worked with the same 12 dermatological images, presented sequentially, providing an initial diagnosis and confidence in that diagnosis. Subsequently, RG students reflected on the case using a structured procedure, while CG students performed a time-filler activity. All students then provided a final diagnosis and confidence in that diagnosis. Outcome measurements were diagnostic accuracy, confidence, and calibration. Results: Reflection increased diagnostic accuracy relative to control (49.7 ± 12.1 vs 38.4 ± 14.6; p = 0.002) but did not affect confidence (64.3 ± 13.2 vs 58.9 ± 20.1; p = 0.228) nor calibration (0.15 ± 0.16 vs 0.20 ± 0.19, p = 0.197). Overall, case difficulty influenced calibration, with students showing more overconfidence on more difficult cases (p <0.001). Conclusions: Deliberate reflection increased diagnostic accuracy in dermatology but did not affect confidence and calibration. Calibration was worse on more difficult cases, suggesting that calibration is a knowledge-related phenomenon.