Assessing variability and uncertainty in orthopedic randomized controlled trials

Abstract

Background and purpose — Low statistical power remains endemic in clinical medicine including orthopedics and manifests as high uncertainty and wide confidence intervals (CI). We evaluated the reporting and correspondence between power calculation and observed data on key parameters of variability and uncertainty in orthopedic randomized controlled trials (RCTs). Material and methods — RCTs with 1:1 allocation published in 8 major orthopedic journals between 2016 and 2017 with one continuous primary outcome were included in the review. The components of power calculation and observed standard deviation (SD), mean difference (MD), and confidence interval (CI) of MD between groups were assessed for primary outcome. Results — 160 RCTs were included, of which 93 (58%) and 138 (86%) studies reported the estimated SD and MD in the power calculation, respectively. The median ratio of the estimated SD and SDs observed in the data was 1.0 (IQR –0.76 to 1.32) for 69 (43%) studies. Only 31 of 138 studies reported the CI of MD in primary outcome. In 42% of the negative studies, the estimated MD was included in the CI of the observed MD. Interpretation — The key parameters of data variability, both in power analyses and in final study results, were poorly reported. Low power in orthopedics may result from too high an estimated effect size due to an overoptimistic estimate of MD between study groups. In almost half of the studies, overlap of the CI of the observed MD and estimated MD suggested that the reported results of these studies were inconclusive.