Comparing Pre-Post Change Across Groups: Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores

Abstract

Psychological researchers often investigate differences between groups in the amount of change from pre-test to post-test. For example, researchers treating a group of depressed students may wish to compare the amount of improvement from pre-intervention to post-intervention for males and females, or the researchers may randomly assign participants to groups and compare their improvement across two treatments. In the first case, there are likely to be pre-test differences between the groups, whereas in the second case no pre-test group differences are expected. Three of the most popular methods for comparing independent groups with regard to the amount of change are difference scores, ANCOVA, and residualized change scores. Although the choice between these methods is sometimes clear, in most instances this is not the case. In this research, a simulation study was used to determine the effect of many common issues on the bias, Type I error rates, and power of the difference score, ANCOVA, and residual change score. These issues included: sample size, reliability of the measure, floor/ceiling effects, effect size, and baseline group differences. Results from the study are used to provide specific recommendations with regards to applying each of these three methods.