Revisiting the retrospective pretest

Abstract

The purpose of the present study was to examine a common practice in some areas of program evaluation, the retrospective pretest, and to present recommendations regarding its use. The authors review literature to emphasize first, that bias is likely in both prospective and retrospective ratings, and second, that under some circumstances, retrospective pretests may introduce greater bias than traditional pretests. The authors examined data from 100 participants who attended a family program at 15 sites. Results supported the hypothesis that items on which parents were asked to endorse socially desirable parenting behaviors resulted in greater discrepancies between prospective and retrospective ratings. Effect sizes calculated from prospective ratings were comparable with published effect sizes from research trials of the same program. The authors conclude that replacing traditional with retrospective pretests does not eliminate bias. They recommend traditional pretests for examination of program effects and retrospective pretests for examination of subjective experiences of program-related change. © 2005 American Evaluation Association.