ANRV364-PS60-13 ARI 27 October 2008 16:18 Prejudice Reduction: What Works? A Review and Assessment of Research and Practice Elizabeth Levy Paluck1 and Donald P. Green2 1Harvard Academy for International and Area Studies, Weatherhead Center for International Affairs, Harvard University, Cambridge, Massachusetts 02138 email: epaluck@wcfia.harvard.edu 2Institution for Social and Policy Studies, Yale University, New Haven, Connecticut 06520-8209 email: donald.green@yale.edu Annu. Rev. Psychol. 2009. 60:339���67 The Annual Review of Psychology is online at psych.annualreviews.org This article���s doi: 10.1146/annurev.psych.60.110707.163607 Copyright c 2009 by Annual Reviews. All rights reserved 0066-4308/09/0110-0339$20.00 Key Words field experiments, evaluation, stereotype reduction, cooperative learning, contact hypothesis, peace education, media and reading interventions, diversity training, cultural competence, multicultural education, antibias education, sensitivity training, cognitive training Abstract This article reviews the observational, laboratory, and field experimental literatures on interventions for reducing prejudice. Our review places special emphasis on assessing the methodological rigor of existing re- search, calling attention to problems of design and measurement that threaten both internal and external validity. Of the hundreds of stud- ies we examine, a small fraction speak convincingly to the questions of whether, why, and under what conditions a given type of interven- tion works. We conclude that the causal effects of many widespread prejudice-reduction interventions, such as workplace diversity train- ing and media campaigns, remain unknown. Although some inter- group contact and cooperation interventions appear promising, a much more rigorous and broad-ranging empirical assessment of prejudice- reduction strategies is needed to determine what works. 339

ANRV364-PS60-13 ARI 27 October 2008 16:18 Prejudice: a negative bias toward a social category of people, with cognitive, affective, and behavioral components Contents INTRODUCTION . . . . . . . . . . . . . . . . . . 340 Scope of the Review. . . . . . . . . . . . . . . . 341 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 NONEXPERIMENTAL RESEARCH IN THE FIELD. . . . . . . . . . . . . . . . . . . 343 Studies with No Control Group . . . . 343 Qualitative Studies . . . . . . . . . . . . . . . . . 344 Cross-Sectional Studies . . . . . . . . . . . . 344 Quasi-Experimental Panel Studies . . 344 Near-Random Assignment . . . . . . . . . 345 Conclusion: Nonexperimental Research . . . . . . . . . . . . . . . . . . . . . . . 345 EXPERIMENTAL RESEARCH CONDUCTED IN THE LABORATORY . . . . . . . . . . . . . . . . . . . 345 Intergroup Approaches . . . . . . . . . . . . . 345 Individual Approaches. . . . . . . . . . . . . . 347 Lessons for the Real World from Laboratory Experiments . . . . . . . . 349 Conclusion: Experimental Research in the Laboratory . . . . . . . . . . . . . . . 351 EXPERIMENTAL RESEARCH CONDUCTED IN THE FIELD. . 351 Cooperative Learning . . . . . . . . . . . . . . 352 Entertainment . . . . . . . . . . . . . . . . . . . . . 353 Discussion and Peer Influence . . . . . . 354 Instruction . . . . . . . . . . . . . . . . . . . . . . . . 354 Less-Frequently Studied Approaches in the Field . . . . . . . . . . . . . . . . . . . . . 355 Lessons of Field Experimental Research . . . . . . . . . . . . . . . . . . . . . . . 356 Recommendations . . . . . . . . . . . . . . . . . 357 DISCUSSION . . . . . . . . . . . . . . . . . . . . . . . 357 Final Thoughts . . . . . . . . . . . . . . . . . . . . 359 INTRODUCTION By many standards, the psychological literature on prejudice ranks among the most impres- sive in all of social science. The sheer volume of scholarship is remarkable, reflecting decades of active scholarly investigation of the mean- ing, measurement, etiology, and consequences of prejudice. Few topics have attracted a greater range of theoretical perspectives. Theorizing has been accompanied by lively debates about the appropriate way to conceptualize and mea- sure prejudice. The result is a rich array of mea- surement strategies and assessment tools. The theoretical nuance and methodologi- cal sophistication of the prejudice literature are undeniable. Less clear is the stature of this lit- erature when assessed in terms of the practi- cal knowledge that it has generated. The study of prejudice attracts special attention because scholars seek to understand and remedy the so- cial problems associated with prejudice, such as discrimination, inequality, and violence. Their aims are shared by policymakers, who spend billions of dollars annually on interventions aimed at prejudice reduction in schools, work- places, neighborhoods, and regions beset by in- tergroup conflict. Given these practical objec- tives, it is natural to ask what has been learned about the most effective ways to reduce preju- dice. This review is not the first to pose this ques- tion. Previous reviews have summarized evi- dence within particular contexts (e.g., the lab- oratory: Wilder 1986 schools: Stephan 1999 cross-nationally: Pedersen et al. 2005), age groups (e.g., children: Aboud & Levy 2000), or for specific programs or theories (e.g., co- operative learning: Johnson & Johnson 1989 intergroup contact: Pettigrew & Tropp 2006 cultural competence training: Price et al. 2005). Other reviews cover a broad range of prejudice- reduction programs and the theories that un- derlie them (e.g., Oskamp 2000, Stephan & Stephan 2001). Our review differs from prior reviews in threerespects.First,thescopeofourreviewisas broad as possible, encompassing both academic and nonacademic research. We augment the lit- erature reviews of Oskamp (2000) and Stephan & Stephan (2001) with hundreds of additional studies. Second, our assessment of the preju- dice literature has a decidedly methodological focus. Our aim is not simply to canvass exist- ing hypotheses and findings but to assess the internal and external validity of the evidence. To what extent have studies established that 340 Paluck �� Green

ANRV364-PS60-13 ARI 27 October 2008 16:18 interventions reduce prejudice? To what ex- tent do these findings generalize to other set- tings? Third, building on prior reviews that present methodological assessments of cultural competence (Kiselica & Maben 1999) and anti- homophobia (Stevenson 1988) program eval- uations, our methodological assessment pro- vides specific recommendations for enhancing the practical and theoretical value of prejudice reduction research. Scope of the Review We review interventions aimed at reducing prejudice, broadly defined. Our purview in- cludes the reduction of negative attitudes to- ward one group (one academic definition of prejudice) and also the reduction of related phenomena like stereotyping, discrimination, intolerance, and negative emotions toward an- other group. For the sake of simplicity, we refer to all of these phenomena as ���preju- dice,��� but in our descriptions of individual interventions we use the same terms as the investigator. By ���prejudice reduction,��� we mean a causal pathway from some intervention to a reduced levelofprejudice.Excluded,therefore,arestud- ies that describe individual differences in prej- udice, as these studies do not speak directly to the efficacy of specific interventions. Our con- cern with causality naturally leads us to place special emphasis on studies that use random as- signment to evaluate programs, but our review also encompasses the large literature that uses nonexperimental methods. Method Over a five-year period ending in spring 2008, we searched for published and unpublished reports of interventions conducted with a stated intention of reducing prejudice or prejudice-related phenomena. We combed online databases of research literatures in psy- chology, sociology, education, medicine, policy studies, and organizational behavior, pairing primary search words ���prejudice,��� ���stereo- Prejudice reduction: a causal pathway from an intervention (e.g., a peer conversation, a media program, an organizational policy, a law) to a reduced level of prejudice type,��� ���discrimination,��� ���bias,��� ���racism,��� ���homophobia,��� ���hate,��� ���tolerance,��� ���reconcil- iation,��� ���cultural competence/sensitivity,��� and ���multicultural��� with operative terms like ���re- duce,��� ���program,��� ���intervention,��� ���modify,��� ���education,��� ���diversity training,��� ���sensitize,��� and ���cooperat���.��� To locate unpublished aca- demic work, we posted requests on several organizations��� email listservs, including the Society for Personality and Social Psychology and the American Evaluation Association, and we reviewed relevant conference proceedings. Lexis-Nexis and Google were used to locate nonacademic reports by nonprofit groups, gov- ernment and nongovernmental agencies, and consulting firms that evaluate prejudice. We examined catalogues that advertise diversity programs to see if evaluations were mentioned or cited. Several evaluation consultants sent us material or spoke with us about their evaluation techniques. Our search produced an immense database of 985 published and unpublished reports writ- ten by academics and nonacademics involved in research, practice, or both. The assem- bled body of work includes multicultural ed- ucation, antibias instruction more generally, workplace diversity initiatives, dialogue groups, cooperative learning, moral and values edu- cation, intergroup contact, peace education, media interventions, reading interventions, in- tercultural and sensitivity training, cognitive training, and a host of miscellaneous techniques and interventions. The targets of these pro- grams are racism, homophobia, ageism an- tipathy toward ethnic, religious, national, and fictitious (experimental) groups prejudice to- ward persons who are overweight, poor, or dis- abled and attitudes toward diversity, reconcili- ation, and multiculturalism more generally. We excluded from our purview programs that addressed sex-based prejudice (the literature dealing with beliefs, attitudes, and behaviors toward women and men in general, as distin- guished from gender-identity prejudices like homophobia). Sex-based inequality intersects with and reinforces other group-based preju- dice ( Jackman 1994, Pratto & Walker 2004), www.annualreviews.org ��� What Works for Prejudice Reduction? 341

ANRV364-PS60-13 ARI 27 October 2008 16:18 but given the qualitatively different nature and the distinctive theoretical explanations for sex-based prejudice and inequality (Eagly & Mlednic 1994, Jackman 1994, Sidanius & Pratto 1999), we believe relevant interven- tions deserve their own review. The resulting database (available at www.betsylevypaluck. com) constitutes the most extensive list of pub- lished and unpublished prejudice-reduction re- ports assembled to date. This sprawling body of research could be organized in many different ways. In order to focus attention on what kinds of valid con- clusions may be drawn from this literature, we divide studies according to research design. This categorization scheme generates three groups: nonexperimental studies in the field, experimental studies in the laboratory, and ex- perimental studies in the field. Supplemental Table 1 (follow the Supplemental Material link from the Annual Reviews home page at http://www.annualreviews.org) provides a descriptive overview of the database according to this scheme. The database comprises 985 studies, of which 72% are published. Nearly two-thirds of all studies (60%) are nonexper- imental, of which only 227 (38%) use a control group. The preponderance of nonexperimen- tal studies is smaller when we look at published work nevertheless, 55% of published studies of prejudice reduction use nonexperimental de- FIELD VERSUS LABORATORY EXPERIMENTS In an experimental design, units of observation (e.g., individuals, classrooms) are assigned at random to a treatment and to placebo or no-treatment conditions. Field experiments are randomized experiments that test the effects of real-world interventions in naturalistic settings, but the distinction between field and lab is often unclear. The laboratory can be the site of very realistic in- terventions, and conversely, artificial interventions may be tested in a nonlaboratory setting. When assessing the degree to which experiments qualify as field experiments, one must consider four aspects of the study: (a) participants, (b) the intervention and its target, (c) the obtrusiveness of intervention delivery, and (d ) the assessed response to the intervention. signs. Of the remaining studies, 284 (29%) are laboratory experiments and 107 (11%) are field experiments (see sidebar Field Versus Labora- tory Experiments). A disproportionate percent- age of field experiments are devoted to school- based interventions (88%). Within each category, we group studies ac- cording to their theoretical approach or inter- vention technique, assessing findings in light of the research setting, participants, and out- come measurement. A narrative rather than a meta-analytic review suits this purpose, in the interest of presenting a richer description of the prejudice-reduction literature. Moreover, the methods, interventions, and dependent vari- ables are so diverse that meta-analysis is poten- tially meaningless (Baumeister & Leary 1997 see also Hafer & Begue ` 2005), especially given that many of the research designs used in this literature are prone to bias, rendering their findings unsuitable for meta-analysis. Our review follows the classification struc- ture of our database. We begin with an overview of nonexperimental prejudice-reduction field research. This literature illustrates not only the breadth of prejudice-reduction interventions, but also the methodological deficiencies that prevent studies from speaking authoritatively to the question of what causes reductions in preju- dice. Next we turn to prejudice reduction in the scientific laboratory, where well-developed the- ories about prejudice reduction are tested with carefully controlled experiments. We examine the theories, intervention conditions, partici- pants, and outcome measures to ask whether the findings support reliable causal inferences about prejudice reduction in nonlaboratory set- tings. We follow with a review of field exper- iments in order to assess the correspondence between these two bodies of research. Because field experiments have not previously been the focus of a research review, we describe these studies in detail and argue that field experi- mentation remains a promising but underuti- lized approach. We conclude with a summary of which theoretically driven interventions seem most promising in light of current evidence, and we provide recommendations for future 342 Paluck �� Green

ANRV364-PS60-13 ARI 27 October 2008 16:18 research (see sidebar Public Opinion Research and Prejudice Reduction). NONEXPERIMENTAL RESEARCH IN THE FIELD Random assignment ensures that participants who are ���treated��� with a prejudice-reduction intervention have the same expected back- ground traits and levels of exposure to out- side influences as participants in the control group. Outcomes in a randomized experiment are thus explained by a quantifiable combina- tion of the intervention and random chance. By contrast, in nonexperimental research the out- comes can be explained by a combination of the intervention, random chance, and unmeasured pre-existing differences between comparison groups. So long as researchers remain uncer- tain about the nature and extent of these biases, nonexperimental research eventually ceases to be informative and experimental methodology becomes necessary to uncover the unbiased ef- fect (Gerber et al. 2004). For these reasons, randomized experiments are the preferred method of evaluation when stakes are high (e.g., medical interventions). Prejudice is cited as a cause of health, eco- nomic, and educational disparities (e.g., Amer- ican Psychological Association 2001), as well as terrorism and mass murder (Sternberg 2003). For scientists who understand prejudice as a pandemic of the same magnitude as that of AIDS or cancer, a reliance on nonexperimen- tal methods seems justifiable only as a short- run approach en route to experimental testing. Nevertheless, in schools, communities, organi- zations, government offices, media outlets, and health care settings, the overwhelming major- ity of prejudice-reduction interventions (77%, or 367 out of the 474 total field studies in our database) are evaluated solely with nonexperi- mental methods, when they are evaluated at all. Studies with No Control Group The majority of nonexperimental field studies do not use a control group to which an inter- PUBLIC OPINION RESEARCH AND PREJUDICE REDUCTION It is ironic but not coincidental that the largest empirical litera- ture on the subject of prejudice���namely, public opinion research on the subject of race and politics���has little, if any, connection to the subject of prejudice reduction. Many of the most impor- tant and influential theories about prejudiced beliefs, attitudes, and actions have grown out of public opinion research. These theories examine the role of preadult socialization experiences (Sears 1988), group interests and identities (Bobo 1988), politi- cal culture and ideology (Sniderman & Piazza 1993), and mass media portrayals of issues and groups (Gilliam & Iyengar 2000, Mendelberg 2001). They diagnose the origins of prejudice, often tracing it to large-scale social forces such as intergroup compe- tition for status and resources, but rarely do they propose or test interventions designed to ameliorate prejudice. Taking prejudice as a fixed personal attribute, this literature instead tends to offer suggestions about how to frame issues (e.g., public spending on welfare) in ways that mitigate the expression of prejudice (e.g., by reminding respondents that most welfare recipients are white). vention group may be compared most eval- uations of sensitivity and cultural-competence programming, mass media campaigns, and di- versity trainings are included in this category. Many no-control evaluations use a postinter- vention feedback questionnaire. For example, Dutch medical students described their expe- riences visiting patients of different ethnici- ties (van Wieringen et al. 2001), and Canadian citizens reported how much they noticed and liked the ���We All Belong��� television and news- paper campaign (Environics Research Group Limited 2001). Other feedback questionnaires ask participants to assess their own change: Diversity-training participants graded them- selves on their knowledge about barriers to suc- cess for minorities and the effects of stereotypes and prejudice (Morris et al. 1996). Other no- control group studies use repeated measure- ment before and after the intervention: We were unable to locate a sensitivity- or diversity- training program for police that used more than a prepost survey of participating officers. Such strategies may reflect a lack of resources for, www.annualreviews.org ��� What Works for Prejudice Reduction? 343