Small Group Statistics: A Monte Carlo Comparison of Parametric and Randomization Tests

Abstract

Under some conditions, behavioral research includes statistical tests to assess the probability of experimental outcomes. Occasionally, sample sizes are insufficient to fulfill the assumptions of traditional parametric procedures. The following study assessed the statistical advantages, correspondence, and accuracy of univariate analyses when small-n samples are calculated in both randomized and traditional/parametric formats. Specifically, we compared probability values for both parametric and randomized two-sample independent t-tests across a series of Monte Carlo, pseudorandom two-group data sets of equal size. In doing so, we identified some patterns of probability change associated with decreasing group size.