Efficient implementation monitoring in routine prevention practice: A grand challenge for schools

Abstract

Objective: Monitoring the implementation of preventive interventions in schools is challenging for educators and school social workers. This study sought to verify the implementation–outcome relationship; determine whether implementer characteristics predict participation in implementation-monitoring protocols; and suggest the number of (a) implementers requiring monitoring, (b) planned observations per implementer, and (c) items rated within each observation that are necessary to monitor program implementation. Method: Data are from a district-wide implementation of the Promoting Alternative Thinking Strategies (PATHS) curriculum. Two technical-assistance providers attempted to make 8 observations of 170 classroom teachers across 15 elementary schools while they delivered the PATHS program. Teacher characteristics, adherence, participant responsiveness, overall implementation quality, and other implementation aspects were observed. Student social–emotional competence was assessed with the Devereux Student Strengths Assessment-Mini at 3 time points to verify the implementation– outcome relationship. Results: Growth of student social–emotional competence was predicted by the overall quality of PATHS implementation. Teacher compliance with implementation-monitoring protocols declined over time, and teachers who demonstrated an initial commitment to implementation participated more fully in monitoring, regardless of their initial implementation quality. A teacher’s initial implementation quality, however, predicted their long-term implementation quality. Items within a single observation were highly intercorrelated. Conclusions: Findings suggest strategies that can be used to improve teacher participation and reduce inefficiencies in implementation-monitoring protocols.