Evaluating Intervention Fidelity and Adaptation Within Context: A Mixed-Methods Study of Implementation Practice Within Public Early Intervention Systems

Abstract

Background: The science of intervention adaptation is rapidly expanding, yet there has been limited research evaluating how context affects intervention fidelity and adaptation. The current study sought to address this gap by closely characterizing the delivery of an autism evidence-based practice (EBP), Project ImPACT, within an Early Intervention (EI) system to understand how context shaped both intervention adaptation and providers’ coaching fidelity. Method: Twenty-one EI providers were trained in Project ImPACT. Following training, providers submitted videos of each of their Project ImPACT sessions, which were scored for Project ImPACT coaching fidelity, Project ImPACT adaptation, and the presence and quantity of supplemental therapeutic content. After each session, EI providers also completed a brief survey about how they delivered Project ImPACT and adaptations they made. Results: Mixed methods data from 100 sessions demonstrated that how providers reported delivering Project ImPACT was misaligned from adaptations that were observed within the same session. Overall, providers’ Project ImPACT fidelity was variable and driven by the integration of other content areas within the confines of relatively short therapy sessions. EI providers adapted Project ImPACT in approximately half of their sessions and spent about 17% of their recorded session time covering other therapeutic content. Spending a greater percentage of session time integrating other content areas was significantly associated with dropping core Project ImPACT coaching activities and having lower Project ImPACT fidelity within that same session. Conclusion: The current study highlights the critical role of context in shaping providers’ Project ImPACT coaching fidelity. Fidelity outcomes in this study were consistent with other EI implementation trials and raise questions about fidelity benchmarks and normative delivery within community settings. Findings also highlight the need for holistic fidelity tools and training models that support the delivery of core intervention functions in relationship to child-, family-, and system-level factors.