New approaches for estimation of effect sizes and their confidence intervals for treatment effects from randomized controlled trials

Abstract

Although Cohen's d and the growth modeling analysis (GMA) d from linear models are common standardized effect sizes used to convey treatment effects, popular statistical software packages do not include them in their standard outputs. This article demonstrated the use of statistical software with user-prescribed parameter functions (e.g., Mplus) to produce d for treatment effects from both classical analysis and GMA-along with their associated standard errors (SEs) and confidence intervals (CIs). A Monte Carlo study was conducted to examine bias in the SE and CI for GMA d obtained with Mplus and found that both estimates were more accurate when calculated by the software with the standard bootstrap than with the delta method, but the delta method estimates were less biased than respective estimates from extant post hoc equations. Thus, users of many statistical software packages (including SAS, R, and LISREL) should obtain d or GMA d and associated CIs directly. Researchers employing less versatile software-and meta-analysts including ds and GMA ds in their syntheses of treatment effects-should continue to use the conventional post hoc equations. Biases in SEs and CIs for effect sizes obtained with them are ignorable and point estimates of d and GMA d are the same whether obtained directly from the software or with post hoc equations. Keywords effect sizes; confidence intervals; multilevel analysis; latent growth models The need for effect sizes that communicate the potency of intervention effects is now well established (Grissom & Kim, 2012). There is also an increasing recognition of the importance of also providing confidence intervals (CIs) for these effect sizes (Cumming, 2013; Odgaard & Fowler, 2010; Preacher & Kelley, 2011). Effect sizes can be unstandardized or standardized (Kelley & Preacher, 2012). Unstandardized effects sizes have an advantage over standardized effect size when making comparisons among findings from different studies that used the same outcome measure because there is no confounding of effect magnitude with sample homogeneity (Baguley, 2009). However, different studies examining the same hypothesis often use varying