Several recent studies of performance appraisal have used the "paper people" approach, in which raters read performance vignettes and then rate the performance of several hypothetical ratees. This approach may lead to systematically different outcomes from studies in which ratings are based on the direct or indirect (e.g., via videotape) observation of ratees' behavior. One hundred and eleven studies published between 1975 and 1984 were grouped into five major substantive categories, and a comparative meta-analysis was used to contrast the outcomes of paper people studies to those of similar studies in which ratings were based on the observation of ratee behavior. Effect sizes were found to be significantly larger in paper people studies, although this difference was not uniform across all research areas. Results are discussed in terms of differences in signal-to-noise ratios across the two methods. Many methods have been used in research on performance appraisal. One dimension for classifying these methods is based on whether they use a "paper people" approach, in which sub-jects read a series of vignettes containing scaled behavior de-scriptions and then rate the performance of one or more hypo-thetical ratees, or an approach that is based on the direct or indirect observation (i.e., via film or videotape) of the behavior of one or more ratees. The latter method has been used in stud-ies ranging from those examining annual perfomance apprais-als conducted for administrative purposes in organizations (e.g., Cleveland & Landy, 1981) to laboratory studies, in which the ratees are actors carrying out performance scripts (e.g., Murphy, Balzer, Kellam, & Armstrong, 1984). The critical difference between the two approaches is that in a paper people study, subjects receive written descriptions of the performance they are to evaluate, whereas in studies involving behavior ob-servation, subjects must observe, interpret, and recall the be-havior of ratees in the process of evaluating their performance. The use of paper people in research on the interview has been strongly criticized (Gorman, Clover, & Doherty, 1978), al-though Dipboye, Stramler, and Fontenelle (1984) noted that this approach in fact simulates several features of the interview process. There are also grounds for criticizing the use of paper people in performance appraisal research. First, the question of whether raters observe or read summaries of the performance they are to evaluate has a substantial impact on the nature of the rating task. For example, raters' ability to accurately observe, encode, and recall performance is a critical concern in much of the recent research on performance appraisal (DeNisi, Meg-The authors wish to thank several colleagues who provided prepubli-cation drafts of papers for our review.
Mendeley saves you time finding and organizing research
Choose a citation style from the tabs below