Validation of a discharge summary term search method to detect adverse events

Abstract

Adverse events are poor health outcomes caused by medical care. Measuring them is necessary for quality improvements, but current detection methods are inadequate. We performed this study to validate a previously derived method of adverse event detection using term searching in physician-dictated discharge summaries. This was a retrospective, chart review study of a random sample of 245 adult medicine and surgery patients admitted to a multicampus academic medical center in 2002. The authors used a commercially available search engine to scan discharge summaries for the presence of 104 terms that potentially indicate an adverse event. Summaries with any of these terms were reviewed by a physician to determine the term's context. Screen-positive summaries had a term that was contextually indicative of an adverse event. We used a two-stage chart review as the gold standard to determine the true presence or absence of an adverse event. The average patient age was 62 years (standard deviation 18.6) and 55% were admitted to a medical service. By gold standard criteria, 48 of 245 patients had an adverse event. Term searching classified 27 cases with an adverse event, with 11 true positives; 218 cases were classified as not having an adverse event, with 181 true negatives. The sensitivity, specificity, and positive and negative predictive values were 0.23 (95% confidence interval [CI] = 0.11-0.35), 0.92 (95% CI = 0.88-0.96), 0.41 (95% CI = 0.25-0.59), and 0.83 (95% CI = 95% 0.77-0.97), respectively. Although the sensitivity of the method is low, its high specificity means that the method could be used to replace expensive manual chart reviews by nurses.