Eliminating Aggregation Bias in Experimental Research: Random Coefficient Analysis as an Alternative to Performing a ‘by-subjects’ and/or ‘by-items’ ANOVA

Abstract

Experimental psychologists routinely simplify the structure of their data by computing means for each experimental condition so that the basic assumptions of regression/ANOVA are satisfied. Typically, these means represent the performance (e.g. reaction time or RT) of a participant over several items that share some target characteristic (e.g. Mean RT for high‐frequency words). Regrettably, analyses based on such aggregated data are biased toward rejection of the null hypothesis, inflating Type‐I error beyond the nominal level. A preferable strategy for analyzing such data is random coefficient analysis (RCA), which can be performed using a simple method proposed by Lorch & Myers (1990). An easy to use SPSS implementation of this method is presented using a concrete example. In addition, a technique for evaluating the magnitude of potential aggregation bias in a dataset is demonstrated. Finally, suggestions are offered concerning the reporting of RCA results in empirical articles. Researchers routinely transform their data in order to satisfy the assumptions of statistical analyses (e.g. regression analysis). For example, log, reciprocal, and square‐root transformations are all used to correct the shape of empirical