Violation of the homogeneity of regression slopes assumption in ANCOVA for two-group pre-post designs: Tutorial on a modified Johnson-Neyman procedure

Abstract

Aptitude-treatment interaction (ATI) effects are present when individuals demonstrate differential outcomes across treatments based upon aptitude—that is, any measurable individual characteristic, attribute, or ability (e.g., anxiety, learning style, motivation, prior knowledge). ATI effects may exist in data from one design commonly used in psychological and educational re-search—the two-group pre-post design—in which pre-intervention scores may be considered to reflect individual aptitude. Researchers may mistakenly overlook these effects, however, due to in-appropriate analytical approaches. When applying analysis of covariance (ANCOVA), it is important to check for ANCOVA assumptions, including an assumption known as homogeneity of regression slopes. When heterogeneity of regression slopes is found, ATI effects are revealed. Consequently, alternative approaches to ANCOVA must be sought. Using formulae based on the Johnson-Neyman procedure to define simultaneous regions of significance is one straightforward alternative. This tu-torial outlines the process for analyzing data resulting from two-group pre-post studies when data violate the ANCOVA assumption of homogeneity of regression slopes. What was initially viewed as an obstacle may result in the discovery of an ATI effect, which may be described statistically through simple mathematical calculations.