Validation of a fidelity rating instrument for consumer-operated services

Abstract

With the emphasis on the use of evidence-based practices has come a need to measure the fidelity of replications to the operations and principles of original models. Recent reviews have focused on methods to develop fidelity measures for evidence-based program models. However, the issue of how to validate such measures has been given scant attention. The research reported here attempted to validate a fidelity rating instrument (FRI) for consumer-operated drop-in centers on the basis of convergent validation methods. That is, fidelity ratings by trained observers were validated in relation to reports about similar concepts from the users of the centers. The authors describe their methods and present results indicating that all but 4 of 14 fidelity variables from the FRI were validated, as hypothesized. The authors discuss the success of this approach as well as its limitations and identify issues in the further development of fidelity measures. © American Evaluation Association.