Practice doesn't always make perfect: A qualitative study explaining why a trial of an educational toolkit did not improve quality of care

Abstract

Background: Diabetes is a chronic disease commonly managed by family physicians, with the most prevalent complication being cardiovascular disease (CVD). Clinical practice guidelines have been developed to support clinicians in the care of diabetic patients. We conducted a pragmatic cluster randomized controlled trial (RCT) of a printed educational toolkit aimed at improving CVD management in diabetes in primary care, and found no effect, and indeed, the possibility of some harm. We conducted a qualitative evaluation to study the strategy for guideline implementation employed in this trial, and to understand its effects. This paper focuses solely on the qualitative findings, as the RCT's quantitative results have already been reported elsewhere. Methods and Findings: All family practices in the province of Ontario had been randomized to receive the educational toolkit by mail, in either the summer of 2009 (intervention arm) or the spring of 2010 (control arm).A subset of 80 family physicians (representing approximately 10% of the practices randomized and approached, with records on 1,592 randomly selected patients with diabetes at high risk for CVD) then took part in a chart audit and reflective feedback exercise related to their own practice in comparison to the guideline recommendations. They were asked to complete two forms (one pre- and one post-audit) in order to understand their awareness of the guidelines pre-trial, their expectations regarding their individual performance pre-audit, and their reflections on their audit results. In addition, individual interviews with thirteen other family physicians were conducted. Textual data from interview transcripts and written commentary from the pre- and post-audit forms underwent qualitative descriptive analysis to identify common themes and patterns. Analysis revealed four main themes: impressions of the toolkit, awareness was not the issue, 'it's not me it's my patients', and chart audit as a more effective intervention than the toolkit. Participants saw neither the toolkit content nor its dissemination strategy to be effective, indicating they perceived themselves to be aware of the guidelines pre-trial. However, their accounts also indicated that they may be struggling to prioritize CVD management in the midst of competing demands for their attention. Upon receiving their chart audit results, many participants expressed surprise that they had not performed better. They reported that the audit results would be an important motivator for behaviour change. Conclusions: The qualitative findings outlined in this paper offer important insights into why the intervention was not effective. They also demonstrate that physicians have unperceived needs relative to CVD management and that the chart audit served to identify shortcomings in their practice of which they had been hitherto unaware. The findings also indicate that new methods of intervention development and implementation should be explored. This is important given the high prevalence of diabetes worldwide; appropriate CVD management is critical to addressing the morbidity and mortality associated with the disease.