Evaluation of validity and reliability of video analysis and live observations to assess trauma team performance

Abstract

Introduction: A trauma resuscitation is dynamic and complex process in which failures could lead to serious adverse events. In several trauma centers, evaluation of trauma resuscitation is part of a hospital’s quality assessment program. While video analysis is commonly used, some hospitals use live observations, mainly due to ethical and medicolegal concerns. The aim of this study was to compare the validity and reliability of video analysis and live observations to evaluate trauma resuscitations. Methods: In this prospective observational study, validity was assessed by comparing the observed adherence to 28 advanced trauma life support (ATLS) guideline related tasks by video analysis to life observations. Interobserver reliability was assessed by calculating the intra class coefficient of observed ATLS related tasks by live observations and video analysis. Results: Eleven simulated and thirteen real-life resuscitations were assessed. Overall, the percentage of observed ATLS related tasks performed during simulated resuscitations was 10.4% (P < 0.001) higher when the same resuscitations were analysed using video compared to live observations. During real-life resuscitations, 8.7% (p < 0.001) more ATLS related tasks were observed using video review compared to live observations. In absolute terms, a mean of 2.9 (during simulated resuscitations) respectively 2.5 (during actual resuscitations) ATLS-related tasks per resuscitation were not identified using live observers, that were observed through video analysis. The interobserver variability for observed ATLS related tasks was significantly higher using video analysis compared to live observations for both simulated (video analysis: ICC 0.97; 95% CI 0.97–0.98 vs. live observation: ICC 0.69; 95% CI 0.57–0.78) and real-life witnessed resuscitations (video analyse 0.99; 95% CI 0.99–1.00 vs live observers 0.86; 95% CI 0.83–0.89). Conclusion: Video analysis of trauma resuscitations may be more valid and reliable compared to evaluation by live observers. These outcomes may guide the debate to justify video review instead of live observations.