Reliability and accuracy of the screening for adverse events in Brazilian hospitals

Abstract

Objective: To analyze the reliability and accuracy of the screening for adverse events (AEs) conducted by nurses taking the assessment by medical residents as the reference. Design: A validation study of the screening phase of a previous retrospective cohort study based on the patient record review that estimated the incidence of AEs (base study). Setting: Three general teaching hospitals in the State of Rio de Janeiro, Brazil. Participants: A subsample of 242 medical records randomly selected from an original sample of 1103 previously evaluated records. Main Outcome Measure(s): A two-step approach was used for analysis: the identification of at least one screening criterion (first endpoint) and the validation of each identified criterion (second endpoint), taking the assessment by medical residents as the reference. Kappa coefficient; simple percentage agreement; sensitivity; specificity; positive and negative predictive values were calculated. Results: The total agreement between medical residents and nurses on the presence of screening criteria was moderate (78.9%, K = 0.55). Specificity (81.6%) was higher than sensitivity (74.4%). Nurses detected more screening criteria that were later confirmed as true AEs (179 vs. 171, respectively). Significant differences in the detection of the screening criteria: 'Other complications', 'Hospitalization injury' and 'Prior admission' were observed. Conclusion: The results suggested a good performance of the nurses in the screening for detection of AE and showed significant differences in relation to detection of specific screening criteria among reviewers. A better understanding of the screening process and the performance of reviewers was provided. © The Author 2012. Published by Oxford University Press in association with the International Society for Quality in Health Care; All rights reserved.