Context as a fundamental dimension of health promotion program evaluation

Abstract

Context can be broadly defined as "the circumstances or events that form the environment within which something exists or takes place" (Encarta, 1999). That 'something' can be health behavior, another health determinant, an intervention, or an evaluation. Each of these events unfolds, not in a vacuum, but in a complex social context which necessarily shapes how the phenomena are manifest, as well as how they may be taken up, resisted or modified. In this chapter we unpack the nature and significance of social context for health promotion practice and evaluation. Drawing on critical realism, we develop a framework for understanding key dimensions of social context that impact on three key levels: the target phenomena (what health promotion practice is seeking to change or enhance), the intervention (how it is received and plays out, its impact), and efforts to evaluate health promotion interventions (we propose that evaluation practice is also embedded in social context). That social context matters is widely recognized and nothing particularly new. Context is identified as a fundamental dimension of program evaluation (Suchman, 1967;Weiss, 1972), and person-environment and program- environment interactions can be traced back to the human ecology work of Broffenbrenner (1977, 1979). Applications of these concepts and ecological systems theory, in various guises, are found in the health promotion literature (see Best et al., 2003; Chu and Simpson, 1994; Green and Kreuter, 2005; Green, Richard and Potvin, 1996; Stokols, 1992, 2000). Although context receives attention in many health promotion texts (Bartholomew, Parcel, Kok, & Gottlieb, 2000; Green & Kreuter, 2005), it is not routinely integrated into or adequately accounted for in most program evaluations. The complexities involved in mapping contextual factors in evaluation pose significant evaluation challenges. Some interventionists and evaluators may lack the necessary theoretical breadth and methodological skills to adequately unpack, theoretically and empirically, how context matters. Nor may they feel they have the 'luxury' of time or breadth of mandate to tackle what may be seen as more challenging conceptual and methodological issues associated with doing so. This chapter identifies some of these challenging issues and proposes a critical realist framework for addressing these lacunae. The overwhelming emphasis within the dominant post-positivist paradigm in health promotion evaluation research has been to treat context as a source of potential confounders that need to be either 'factored in' (as variables that apply across cases) or 'factored out' ('controlled for' statistically or through study design such as randomization). Identification of 'best practices' that can be disseminated across space and time with predictable outcomes following the results of promising pilot research, also treats context as something of a nuisance to be addressed only insofar as it threatens to seriously compromise implementation fidelity or program outcomes. Further, following Malpas (2003), we believe that increasingly dominant managerial regimes that privilege efficiency and tight fiscal and legal accountability in health and social service delivery seek to tighten administrative control through the standardization of practice. Standardization accords only grudging acknowledgement to the difference that context makes. The inherent 'messiness', unpredictability, and uniqueness of context is difficult to reconcile with an administrative rationality intent on procedural standardization. In short, epistemological, political, and administrative factors have conspired to either obscure the relative importance of social context to program design, implementation, and evaluation or, at the very least, leave largely unexamined or unexplained the ways in which context matters. From studies of small area variations in healthcare practice (Wennberg & Gittelsohn, 1973), to studies of community-based health promotion interventions (Bracht, 1990; Minkler, 1990, 1997), the evidence that context matters is increasingly difficult to ignore. In some fields, such as tobacco control, there is growing awareness that the failure to sufficiently understand the social context of smoking has compromised the field's success record (Flay & Clayton, 2003; Poland et al., 2006). The social distribution of smoking has changed, and thus the social distance between target populations and interventionists, whose assumptions and world view are reflected in programming (Poland et al., 2006). The popularity of a settings approach in health promotion reflects, in part, an understanding of the importance of aligning program design and intervention activities with the realities of the setting for which they're intended (Chu & Simpson, 1994; Dooris et al., 2007; Mullen et al., 1995; Poland, Green, & Rootman, 2000, Poland, Lehoux, Holmes, & Andrews, 2005; Whitelaw et al., 2001). For example, considerable expertise has emerged in school-based health promotion with respect to the essential features of schools, as well as variability in their expression (e.g., inner city versus rural), that impact on program delivery and outcomes. The identification of aspects of context that impact on practice has also been undertaken with respect to community-based programming, workplace health promotion, and interventions tailored for other settings such as hospitals, Aboriginal communities, and prisons, among others. Context is fundamental to understanding the adequacy of program conceptualization and design: do interventions adequately address the social context within which target phenomena, such as health behaviors, are created, sustained and socially distributed in time and space? Context is also fundamental to program implementation and outcomes: are interventions optimized to take advantage of the unique confluence of opportunities available in each local context and which intervention components produce which results under what conditions? Finally, context shapes the production and utilization of evaluation findings: the influence of key assumptions and stakeholders on the design and implementation of the evaluation, as well as the impact of timing and other factors on research uptake. The organization of this chapter reflects the ways in which social context is implicated at three overlapping levels: (a) the nature of the phenomena that are the object of health promotion intervention (the social context of target phenomena); (b) interventions themselves (the social context of health promotion practice); and (c) knowledge development and utilization (the social context of evaluation research). At this juncture it is worth clarifying what we mean by evaluation. We adopt the definition proposed by Rossi and Freeman (1985, p. 19): "the systematic application of social research procedures in assessing the conceptualization and design, implementation, and utility of social intervention programs". We prefer this over less comprehensive definitions because it explicitly makes room for a critique of the adequacy of program conceptualization and design, whereas many evaluation definitions do not and are restricted to determining the extent to which intended outcomes are achieved. The premise of this chapter is that although context is of inescapable importance in health promotion program evaluation, better conceptual, theoretical, and methodological tools are needed to reposition it at the centre of evaluation efforts. Following a review of each of the three layers of context identified above, we draw on diverse disciplinary perspectives to assemble some of the conceptual, theoretical, and methodological tools necessary for a deeper and more satisfying treatment of context in health promotion program evaluation. In particular, we draw on critical social theory and critical realist perspectives to fashion an understanding of how social relations (at the heart of any social intervention) function in different social contexts, for these are critical to understanding how context matters. © 2008 Springer Science+Business Media, LLC.