Comparing the predictive validity of DUI risk screening instruments: Development of validation standards

Abstract

Aims. This study compares the predictive efficacy of driving under the influence (DUI) screening instruments validated in previous studies, illustrates how variations in base rates of failure and selection ratios affect conclusions concerning the efficacy of different instruments, and develops evaluation standards to ensure valid comparisons of risk prediction instruments. Design. The study: (1) examines a sample of 4815 DUI offenders to illustrate how variations in base rates of failure and selection ratios affect traditional measures of predictive efficacy, (2) uses such measures to compare the predictive efficacy of 10 instruments validated in previous studies, and (3) demonstrates the use of a measure of predictive efficacy which is relatively insensitive to the aforementioned variations. Findings. While three instruments examined at specific cut-points consistently ranked highest on several measures of predictive efficacy, use of different evaluation standards produced substantively different conclusions regarding the efficacy of different instruments. Based on the analyses, standards for validation of risk prediction instruments were developed. Conclusions. The findings illustrate how failure to use equivalent standards have led to erroneous conclusions concerning the relative predictive efficacy of different risk prediction instruments. The standards developed in this study should facilitate equivalent comparisons of the predictive efficacy of risk prediction instruments.