Evaluating public health strategies for climate adaptation: Challenges and opportunities from the climate ready states and cities initiative

Abstract

Evaluation generates critical evidence about the effectiveness of health-focused climate adaptation, who receives what benefits, and how to improve program quality. However, using evaluation to improve climate adaptation outcomes with timeliness and context-specificity is uniquely challenging. The United States Centers for Disease Control and Prevention supports health departments to implement adaptation initiatives through the Climate-Ready States and Cities Initiative (CRSCI) grant and minimize negative health impacts of climate change via the Building Resilience Against Climate Effects (BRACE) framework, which includes evaluation. In this paper, we present current evaluation practice by describing the health-focused adaptation actions that were evaluated among CRSCI recipients, the evaluation approaches they used, and the barriers and enablers encountered. Using a mixed methods approach, we abstracted annual progress report and standardized performance measure data to characterize evaluation activities across 18 grant recipients using basic quantitative descriptive analysis. Through structured interviews with three grant recipients and standard team-based qualitative coding and analysis techniques, we developed qualitative case studies to explore evaluation barriers and enablers in richer context. Recipients reported 76 evaluations over the reporting period (2018–2021). Evaluated programs commonly addressed extreme weather (50.0%), followed by heat (27.6%). The most common purpose was to monitor implementation or improve delivery (57.9%). Case studies highlighted barriers to successful evaluation such as limited specialized evaluation expertise and staff time. Enablers included staff motivation to justify program expansion, strong relationships with community partners, and use of evaluation plans. Case studies revealed diverse strategies to seek input from stakeholders disproportionately impacted by climate change and to reduce health disparities. The experiences of CDC grant recipients provide an opportunity to better understand the barriers and enablers of climate and health evaluation practice and to identify practical strategies to expand the value of evaluation in this nascent field.