Designs for Evaluating the Community-Level Impact of Comprehensive Prevention Programs: Examples from the CDC Centers of Excellence in Youth Violence Prevention

Abstract

This article discusses the opportunities and challenges of developing research designs to evaluate the impact of community-level prevention efforts. To illustrate examples of evaluation designs, we describe six projects funded by the Centers for Disease Control and Prevention to evaluate multifaceted approaches to reduce youth violence in high-risk communities. Each of these projects was designed to evaluate the community-level impact of multiple intervention strategies to address individual and contextual factors that place youth at risk for violent behavior. Communities differed across projects in their setting, size, and how their boundaries were defined. Each project is using multiple approaches to compare outcomes in one or more intervention communities to those in comparison communities. Five of the projects are using comparative interrupted time-series designs to compare outcomes in an intervention community to matched comparison communities. A sixth project is using a multiple baseline design in which the order and timing of intervention activities is randomized across three communities. All six projects are also using regression point displacement designs to compare outcomes within intervention communities to those within broader sets of similar communities. Projects are using a variety of approaches to assess outcomes including archival records, surveys, and direct observations. We discuss the strengths and weaknesses of the designs of these projects and illustrate the challenges of designing high-quality evaluations of comprehensive prevention approaches implemented at the community level.