Addressing the Treatment-as-Assigned Assumption in Field Experiments: Lessons Learned From the Birmingham South Saturated Problem-Oriented Policing Hot Spots Experiment

Abstract

This study investigates the effectiveness of increasing police presence in hot spots typically managed with problem-oriented policing (POP) compared to using POP alone. Our block-randomized controlled trial reveals that overall, “saturated POP” does not significantly reduce crime compared to a POP-only approach. However, a more complex pattern emerged across different blocks, namely, an inverse relationship between the level of police presence and the level of recorded crime, regardless of the random assignment. Crime rates decreased in the experimental arm within two statistical blocks, where the assigned dosage of saturated POP was greater than in the control arm. However, in one block, the assigned control hot spots received a higher dosage of police presence than the assigned treatment hot spots, resulting in significantly lower crime rates in the control hot spots. These findings underscore the necessity of reporting field experiment results based on both treatment-as-assigned and some form of treatment-as-delivered models.