Assessment of inter-observer reliability of two five-level triage and acuity scales: A randomized controlled trial

Abstract

Introduction: The Emergency Severity Index (ESI©) is an initial measure of patient assessment in the emergency department (ED). It rates patients based on acuity and predicted resource intensity from Level 1 (most ill) to Level 5 (least resource intensive). Already implemented and evaluated in several US hospitals, ESI has yet to be evaluated in a Canadian setting or compared with the five-level Canadian Emergency Department Triage and Acuity Scale (CTAS). Objective: To compare the inter-observer reliability of 2 five-level triage and acuity scales. Methods: Ten triage nurses, who had all been trained in the use of CTAS, from 4 urban, academic Canadian EDs were randomly assigned either to training in ESI version 3 (ESI v.3) or to refresher training in CTAS. They independently assigned triage scores to 200 emergency cases, unaware of the rating by the other nurses. Results: Number of years of nursing practice was the only significant demographic difference found between the 2 groups (p = 0.014). A quadratically weighted kappa to measure the inter-observer reliability of the CTAS group was 0.91 (0.90, 0.99) and not significantly different from that of the ESI group 0.89 (0.88, 0.99). An inter-test generalizability (G) study performed on the variance components derived from an analysis of variance (ANOVA) revealed G(5) = 0.90 (0.82, 0.99). Conclusions: After 3 hours of training, experienced triage nurses were able to perform triage assessments using ESI v.3 with the same inter-observer reliability as those with experience and refresher training in using the CTAS.