Figuring out fidelity: A worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies

Abstract

Background: In this paper, we identify and respond to the fidelity assessment challenges posed by novel contextualised interventions (i.e. interventions that are informed by composite social and psychological theories and which incorporate standardised and flexible components in order to maximise effectiveness in complex settings). We (a) describe the difficulties of, and propose a method for, identifying the essential elements of a contextualised intervention; (b) provide a worked example of an approach for critiquing the validity of putative essential elements; and (c) demonstrate how essential elements can be refined during a trial without compromising the fidelity assessment. We used an exploratory test-and-refine process, drawing on empirical evidence from the process evaluation of Supporting Policy In health with Research: an Intervention Trial (SPIRIT). Mixed methods data was triangulated to identify, critique and revise how the intervention's essential elements should be articulated and scored. Results: Over 50 provisional elements were refined to a final list of 20 and the scoring rationalised. Six (often overlapping) challenges to the validity of the essential elements were identified. They were (1) redundant-the element was not essential; (2) poorly articulated-unclear, too specific or not specific enough; (3) infeasible-it was not possible to implement the essential element as intended; (4) ineffective-the element did not effectively deliver the change principles; (5) paradoxical-counteracting vital goals or change principles; or (6) absent or suboptimal-additional or more effective ways of operationalising the theory were identified. We also identified potentially valuable 'prohibited' elements that could be used to help reduce threats to validity. Conclusions: We devised a method for critiquing the construct validity of our intervention's essential elements and modifying how they were articulated and measured, while simultaneously using them as fidelity indicators. This process could be used or adapted for other contextualised interventions, taking evaluators closer to making theoretically and contextually sensitive decisions upon which to base fidelity assessments.