A Comparative Simulation Study of the Fairness and Accuracy of Predictive Policing Systems in Baltimore City

Abstract

There are ongoing discussions about predictive policing systems being unfair, for example, by exhibiting racial bias. Law enforcement in some cities, such as Los Angeles, California, and Baltimore, Maryland, have initiated the integration of these systems into their decision-making processes, and some of these systems were advertised as being unbiased. However, later studies discovered that these methods could also be unfair due to feedback loops and being trained on historically biased recorded data. Comparative studies on predictive policing systems are few and insufficiently comprehensive. Crucially, the relative fairness of predictive policing methods with regard to traditional hot spot-based policing has not been established. Moreover, the relationship between fairness and accuracy is complex and requires further study. Furthermore, the case of Baltimore City, Maryland, USA, has not yet been systematically analyzed despite its relevance as an early adopter of predictive policing technologies with a fraught history of social justice concerns around policing. An improved understanding of these questions could better inform policy decisions around predictive policing technologies both in Baltimore and beyond. Therefore, in this work we perform a comprehensive comparative simulation study on the fairness and accuracy of predictive policing technologies in Baltimore. Our results suggest that the situation around bias in predictive policing is more complex than previously assumed. While we find that predictive policing exhibits bias due to feedback loops, as previously reported, we also find traditional hot spot-based policing to have similar issues. Although predictive policing is found to be more fair and accurate than hot spot policing in the short term, it also amplifies bias more quickly, suggesting the potential for worse long-run behavior. In Baltimore, the bias in these systems tended toward over-policing White neighborhoods in some cases, unlike in previous studies. However, when the analysis was restricted to some specific crime types, this tendency differed. Overall, this work demonstrates a methodology for city-specific evaluation and compares behavioral tendencies of predictive policing systems, showing how such simulations can reveal inequities and long-term tendencies. We recommend that authorities and community stakeholders use simulation methodologies to assist in collaboratively navigating the complexities around fairness in predictive policing.