The effect of trial size and variability on statistical power

Abstract

A computer model was developed, validated, and used in conjunction with Monte Carlo procedures to study the effects of sample size (subjects and trials), mean differences, and subject variability on statistical power. Also examined were the differences between single subject (SS) and group results. Mean differences were varied from 1/4 to 4 times the distribution SD resulting in improved power values. Mean group F results ranged from 63.6% to 100% while SS results were poorer, especially for the smaller mean differences (16.8%-100%). Subject variability was examined for a Simple model and two Complex (MOD1 and MOD4) models. MOD1 produced group results similar to the corresponding Simple model with an overall mean of 78.2% and a Complex/Simple (C/S) ratio of 0.99. The more variable model (MOD4) produced fewer significant results (52.9%) and a lesser C/S ratio (0.82). The SS results were more dramatic. The percentages of significant values were less (38.1% and 33.9%) and the C/S ratios favored the Complex models (1.48 and 3.17). Both sample size and trial size had a major impact on the results. In summary, these results provide additional insight into the interactive effects and importance of the factors investigated, especially in the area of SS experiments. © 1995 by the American College of Sports Medicine.