Accuracy and reliability of injury coding in the national Dutch Trauma Registry

Abstract

Objective: Injury coding is well known for lack of completeness and accuracy. The objective of this study was to perform a nationwide assessment of accuracy and reliability on Abbreviated Injury Scale (AIS) coding by Dutch Trauma Registry (DTR) coders and to determine the effect on Injury Severity Score (ISS). Additionally, the coders' characteristics were surveyed. Methods: Three fictional trauma cases were presented to all Dutch trauma coders in a nationwide survey (response rate 69%). The coders were asked to extract and code the cases' injuries according to the AIS manual (version 2005, update 2008). Reference standard was set by three highly experienced coders. Summary statistics were used to describe the registered AIS codes and ISS distribution. The primary outcome measures were accuracy of injury coding and inter-rater agreement on AIS codes. Secondary outcome measures were characteristics of coders: profession, work setting, experience in injury coding and training level in injury coding. Results: The total number of different AIS codes used to describe 14 separate injuries in the three cases was 89. Mean accuracy per AIS code was 42.2% (range 2.4-92.7%). Mean accuracy on number of AIS codes was 23%. Overall inter-rater agreement per AIS code was 49.1% (range 2.4-92.7%). The number of assigned AIS codes varied between 0 and 18 per injury. Twenty-seven percentage of injuries were overlooked. ISS was correctly scored in 42.4%. In 31.7%, the AIS coding of the two more complex cases led to incorrect classification of the patient as ISS < 16 or ISS ≥ 16. Half (47%) of the coders had no (para)medical degree, 26% were working in level I trauma centers, 37% had less than 2 years of experience and 40% had no training in AIS coding. Conclusions: Accuracy of and inter-rater agreement on AIS injury scoring by DTR coders is limited. This may in part be due to the heterogeneous backgrounds and training levels of the coders. As a result of the inconsistent coding, the number of major trauma patients in the DTR may be over- or underestimated. Conclusions based on DTR data should therefore be drawn with caution.