How many subjects? Statistical power analysis in research

Abstract

This month we touch on a fundamental issue in statistical evaluation that often gets overlooked. Testing assumptions for parametric analysis is a fundamental step and a necessary one. For example, let's consider some of the simplest experimental design analysis available; the independent t-test and analysis of variance F test—testing for mean differences among independent groups. These tests have three key assumptions; normality, independence of observations, and homogeneity of variances (HOV). Generally speaking, experimental design dictates random sampling from a well defined population and random assignment to groups (aka. conditions, levels, etc.), both of which should help take care of assumptions mentioned. But, let's focus our attention on the third assumption (HOV), which needs to be (and can be) tested to ensure accurate or valid interpretation of the mean differences. Luckily, most statistical software packages offer a way to test for HOV (including PASW/SPSS). Generally, the Levene's test is used to statistically test the amount of difference between variances (of groups selected for a t-test or F test). Means vs. variances, a Royal Rumble… Levene's test is testing for differences among our group's (2 or more) variances. A t-test is testing for differences among 2 group's means. An F test (one-way ANOVA) is testing for differences among more than 2 group's means. In these contexts, the independent variable is comprised of multiple groups; for the t-test, there are two groups; for the F test, there are more than two groups. Each group represents a treatment or lack of one in the case of a placebo. In essence, each group receives something different as stimulus, for example different drugs administered in each condition of an efficacy study. Essentially, whether looking at 2 groups (t-test) or more than 2 groups (F test), we are concerned with the assumption of homogeneity of variances (among other assumptions). Recall that variance is a measure of dispersion, how much do the scores (of one group) VARY around the mean (whatever that mean happens to be). Mean is a measure of central tendency; arithmetic average. The HOV assumption states that our groups are similar in essence (similar variances), regardless of independent variable level (treatment or condition administered).