Safer delivery of surgical services: a programme of controlled before-and-after intervention studies with pre-planned pooled data analysis

Abstract

BACKGROUND High rates of iatrogenic harm have been confirmed in observational studies of surgery. Most interventions designed to reduce this have been targeted at either workplace culture or operational systems. We hypothesised that an integrated intervention addressing both culture and system might be more effective than either approach alone. OBJECTIVE To evaluate interventions designed to improve surgical team performance by impacting culture or systems in isolation or combination. DESIGN Five controlled intervention experiments, addressing system, culture or both, were performed in operating theatres. A final whole-system intervention study integrated approaches that showed benefit in these experiments. The five linked studies were subjected to a pre-planned pooled analysis to identify the effects of interventions, combinations and confounders. A qualitative interview study provided explanatory data on the mechanisms of intervention success and failure. An economic analysis was conducted. SETTING Operating theatres in five hospitals, performing orthopaedic, trauma, vascular and plastic surgery were used for the linked studies. The final study occurred in a tertiary referral neurosurgery unit. PARTICIPANTS The main study subjects were clinical staff. Patient outcomes, both clinical and patient reported, were collected as secondary outcome measures. INTERVENTIONS The interventions tested were (1) teamwork training (TT) based on the aviation crew resource management model, (2) the development of a set of standard operating procedures (SOPs), (3) a safety improvement programme based on lean principles, (4) TT plus SOPs and (5) TT plus lean. The final intervention used elements of all three strategies. MAIN OUTCOME MEASURES Primary outcomes were team non-technical skills [as measured by the Oxford Non-Technical Skills (NOTECHS) II scale score] and team technical performance (via the ‘glitch count’). Secondary outcomes were compliance with the World Health Organization (WHO)’s checklist procedures, patient length of stay, readmissions, 30-day mortality, complications and patient-reported outcome measures [as measured by the European Quality of Life-5 Dimensions (EQ-5D)]. A qualitative interview study provided explanatory data on the mechanisms of intervention success and failure. An economic analysis was conducted. DATA SOURCES Direct observation of whole operations, clinical records, hospital information systems and EQ-5D questionnaires. The qualitative study used semistructured interviews. STATISTICAL METHODS Individual studies were analysed using two-way analysis of variance, and an overall individual patient pooled analysis was performed. Methods validation studies and other analyses used chi-squared test, correlation and regression methods as appropriate. RESULTS We studied 453 operations. The results of single interventions were inconsistent. TT alone improved non-technical skills and WHO compliance (p < 0.001) but not technical performance, whereas the systems interventions (lean and SOP) improved non-technical skills and technical performance (p < 0.001), but were less effective in improving WHO compliance. The integrated intervention approaches improved all aspects of team performance except time-out attempt rate, whereas the single approaches were significantly poorer at improving checklist compliance (p < 0.001) and failed to improve glitch rate. Combining all three strategies did not increase the percentage of successful projects. The qualitative analysis confirmed that integrated interventions better addressed the breadth of challenges that face surgical safety but also indicated that differences in implementation between integrated- and single-intervention studies amplified their differential effect. CONCLUSIONS A combination of TT plus systems improvement training appears more effective in improving team performance than either approach alone. An implementation strategy based on an understanding of the barriers to change in hospitals is important for success. FUTURE WORK More work is required to understand and measure barriers to safety improvement. Implementation strategies need to be tested empirically. Methods for delivering integrated interventions on a larger scale need development. A cluster randomised trial of the integrated-systems/culture-improvement approach is warranted. FUNDING The National Institute for Health Research Programme Grants for Applied Research programme.