Evaluating the Performance of Paper People

Abstract

Several recent studies of performance appraisal have used the "paper people" approach, in which raters read performance vignettes and then rate the performance of several hypothetical ratees. This approach may lead to systematically different outcomes from studies in which ratings are based on the direct or indirect (e.g., via videotape) observation of ratees' behavior. One hundred and eleven studies published between 1975 and 1984 were grouped into five major substantive categories, and a comparative meta-analysis was used to contrast the outcomes of paper people studies to those of similar studies in which ratings were based on the observation of ratee behavior. Effect sizes were found to be significantly larger in paper people studies, although this difference was not uniform across all research areas. Results are discussed in terms of differences in signal-to-noise ratios across the two methods. © 1986 American Psychological Association.