Evaluation in the Real World: Decision Points and Rationales in Creating A Rigorous Study Designed to Convey Ecologically Valid Findings

Abstract

Rigorously evaluating community-based interventions for multiply marginalized populations is fraught with challenges under the best of circumstances. This manuscript describes the methodology chosen to evaluate an innovative model designed to help survivors of intimate partner violence obtain safe and stable housing. We justify the choice of evaluation design from a community psychology perspective and detail why we believe the multi-method, multi-source design, that also focuses on social context, will maximize ecological validity and, therefore, propel the scale-up of the intervention if it is found to be effective. Longitudinal data are being collected from program recipients over time, the advocates who worked with them, agency service records, and monthly documentation of agency resources on hand that can impact services provided. Special attention is focused on capturing contextual information that can impact program success. While randomized control trials are still too often heralded as “the gold standard” for measuring intervention effectiveness, we maintain that the current design, which was developed in partnership with key community stakeholders, holds more promise when evaluating many community-based programs.