Now You See It, Now You Don’t: A Simulation and Illustration of the Importance of Treating Incomplete Data in Estimating Race Effects in Sentencing

Abstract

Objectives: Evaluate the impact of missing data on observed racial disparities in the likelihood of an incarceration sentence, given that complete case analysis in the common analytic approach used in criminological research. Methods: Using a simulation study with data based on cases sentenced in the Court of Common Pleas in Pennsylvania between 2010 and 2019, we assess the differences in the likelihood of incarceration between similarly situated White and Black defendants based on varying sample sizes and patterns of missing data. Results: Complete case analysis (CCA) of incomplete data can fail to provide unbiased estimates of the race effect, even with less than 10% of cases missing. The degree of bias introduced depends on the amount, pattern, assumptions, and treatment of missing data. Multiple imputation provides an established, valid methodology for the unbiased estimation of race effects when data are missing at random, and this holds across sample sizes and number of imputations. Conclusions: The existence and magnitude of race effects on the likelihood of an incarceration sentence can vary greatly based on the degree, pattern, assumptions, and treatment of missing data. Limitations include that missing data mechanisms cannot be truly known outside of a data simulation. Future sentencing research should prioritize the identification, treatment, and reporting of missing data prior to isolating race effects, in line with calls from the field for more open science practices. Sensitivity analyses should also be prioritized.