There is more to methodology than method

Abstract

There is much more to evaluation than collecting, analyzing, and interpreting scientific data in order to compare the outcomes of vious treatments. For the past 40 years, method-related discussions in the field of program evaluation have evolved to include models and reflections on the complex and multiple roles associated with the practice of evaluation. In fact, for Shadish, Cook, & Leviton, (1991), the knowledge basis which pertains to the practice of evaluation must consider issues related to the evaluator's roles as well as to the design of evaluation. Thus, evaluation practice requires both the methodological and technical competencies for systematic inquiry, in addition to a whole set of interpersonal and negotiation skills, identified by Brown (1995) as ranging from pedagogical to political. Conceiving evaluation as a practice, as we do in this collection of essays, is based upon the premise that evaluators are more than good and rigorous scientists, implementing empirical inquiry devices to study programs and interventions.While there is clearly more to the evaluator's role than data-related activities, how do these two aspects of practice which are inherently part of program evaluation, come together and build knowledge of evaluation practice? To address this question, we propose revisiting the teleological, epistemological and ontological foundations uponwhich evaluation roles are defined. Consistent with the orientation of this book, we define evaluators' roles vis-avis a program as framed by one's evaluation practice. We suggest that evaluation practice does not however simply represent a repertoire of roles from which the evaluator may (more or less) arbitrarily choose from in order to define themselves and their evaluation activities (i.e. their methodological tool kit). In this chapter we argue that the evaluator's role vis-à-vis programs stakeholders and the approach taken to identify, describe, and measure a program and its effects, form an evaluator's practice, which is consistent and coherent across the contexts of their work. We will also argue that a practice, like a paradigm, constitutes an organized rationality (Crozier & Friedberg, 1977) common to groups of individuals that allow those groups to identify, structure, interpret and solve practical problems identifiable through the lens of a practice or paradigm (Kuhn, 1962). © 2008 Springer Science+Business Media, LLC.