Validity and reliability of the novel three-item occupational violence patient risk assessment tool

Abstract

Aim: To develop and psychometrically test an occupational violence (OV) risk assessment tool in the emergency department (ED). Design: Three studies were conducted in phases: content validity, predictive validity and inter-rater reliability from June 2019 to March 2021. Methods: For content validity, ED end users (mainly nurses) were recruited to rate items that would appropriately assess for OV risk. Subsequently, a risk assessment tool was developed and tested for its predictive validity and inter-rater reliability. For predictive validity, triage notes of ED presentations in a month with the highest OV were assessed for presence of OV risk. Each presentation was then matched with events recorded in the OV incident register. Sensitivity and specificity values were calculated. For inter-rater reliability, two assessors—trained and untrained—independently assessed the triage notes for presence of OV risk. Cohen's kappa was calculated. Results: Two rounds of content validity with a total of N = 81 end users led to the development of a three-domain tool that assesses for OV risk using aggression history, behavioural concerns (i.e., angry, clenched fist, demanding, threatening language or resisting care) and clinical presentation concerns (i.e., alcohol/drug intoxication and erratic cognition). Recommended risk ratings are low (score = 0 risk domain present), moderate (score = 1 risk domain present) and high (score = 2–3 risk domains present), with an area under the curve of 0.77 (95% confidence interval 0.7–0.81, p