Assessment of the reliability of the IHI Global Trigger Tool: New perspectives from a Brazilian study

Abstract

Objective: To assess the reliability of the Institute for Healthcare Improvement's Global Trigger Tool (IHI-GTT) between nurses and medical students as primary reviewers to measure adverse events (AEs). Design: Interrater reliability study. Setting: A 500-bed general public hospital in Belo Horizonte, Brazil. Participants: A randomly selected sample of 220 hospital admissions of adults (≥18 years) from Oct-Nov, 2016. Intervention: Two 4th-5th year-medical students and two experienced nurses applied a Portuguese-translated version of the IHI-GTT to medical records. The role of medical reviewer was performed by two senior physicians specialists in Internal Medicine. Main Outcome Measures: Ability to identify AEs was compared between pairs and against medical reviewer through percentage inter-examiner agreement and Kappa coefficient (K). Two outcomes - "AE identification"and "category of harm"- were evaluated according to two different denominators - "admissions"(the total number of admissions evaluated in the sample; reflects the presence or not of at least one AE in each admission) and "all possibilities of agreement"(obtained by adding each identified AE to the admissions without events; allows agreement assessment to be performed for each AE individually). Results: Were identified 199 adverse events in 90 hospitalizations, with rates of 40.9% of admissions with AEs, 76.1 AEs/1,000 patient-days and 90.5 AEs/100 admissions. Comparing student-pair and nurse-pair, we found K = 0.76 (95% IC 0.62-0.88) and K = 0.17 (95% IC 0.06-0.27) for "AE identification"outcome and K = 0.28 (95% IC 0.01-0.55) and K = 0.46 (95% IC 0.28-0.64) for "category of harm"outcome to denominators "admission"and "all possibilities of agreement", respectively. There was no significant difference between the performances of the different primary reviewers composed in any analyses. Conclusion: IHI-GTT reliability varies considerably depending on the denominator used to calculate agreement. As the purpose of the tool is, in addition to measuring, promoting opportunities for quality of care improvement, the individual analysis of the AEs seems more appropriate. Further studies are needed to assess the implications of the slight agreement reached between primary reviewers on the test's overall accuracy. Moreover, advanced medical students may be considered for primary review in settings where unavailability of staff is a barrier to IHI-GTT adoption.