Tracing scientific reasoning in psychiatry: Reporting of statistical inference in abstracts of top journals 1975–2015

Abstract

Objective: To analyze the reporting of statistical inference in psychiatry. Method: We searched 63,928 abstracts, published in 15 leading psychiatric journals (1975–2015). Results: Median abstract length increased from 664 (1975) to 1,323 (2015) characters, and median use of numbers from two to 14/abstract. A total of 3.6% of all abstracts exclusively contained significance terminology in a nonstatistical sense, and 45% showed some form of statistical inference, increasing from 26% to 52%. In those abstracts, statistical inference based on thresholds was dominant. Its proportion decreased from 99% to 66%, but with rising numbers of articles, figures rose from 1,095 to 2,382. Although reporting p values without thresholds did not appear 40 years ago and remains rare, combining precise p values with thresholds is now common. In 2010–2015, 86% of abstracts contained p values or p value thresholds, 22% confidence intervals, and 7% confidence intervals only. Results varied across journals. Conclusion: There is a moderate shift from reporting p values along set thresholds, such as p ≤ 0.05, to presenting precise p values and confidence intervals, but not as pronounced as in epidemiology and general medicine. The long debate on estimation over testing has not led to a substantial replacement of p values by confidence intervals. Null hypothesis testing (“p ≤.05”) dominates statistical thinking.