The Impact of APA and AERA Guidelines on Effect Size Reporting

Abstract

Given the long history of effect size (ES) indices (Olejnik and Algina Contemporary Educational Psychology, 25, 241-286 2000) and various attempts by APA and AERA to encourage the reporting and interpretation of ES to supplement findings from inferential statistical analyses, it is essential to document the impact of APA and AERA standards on ES reporting practices. In this paper, we investigated the impact by examining findings from 31 published reviews and our own review of 451 articles published in 2009 and 2010. The 32 reviews were divided into two periods: before and after 1999. A total of 116 journals were reviewed. Findings from these 32 reviews revealed that since 1999, the ES reporting has improved in terms of its rate, variety, interpretation, confidence intervals, and fullness. Yet several inadequate practices still persisted: (1) the dominance of Cohen's d, and the unadjusted R2/η2, (2) the mere labeling of ES, (3) the under-reporting of confidence intervals, and (4) a lack of integration between ES and statistical tests. The paper concludes with resources on Internet and recommendations for improving ES reporting practices. © 2013 Springer Science+Business Media New York.