Publication language and the estimate of treatment effects of physical therapy on balance and postural control after stroke in meta-analyses of randomised controlled trials

Abstract

Background Findings regarding the impact of language bias on treatment effect estimates (TEE) are conflicting, and very few studies have assessed these impacts in rehabilitation. The purpose was to compare TEE between studies published in non-English language (SPNEL) and those published in English language (SPEL) included in a previously published meta-analysis assessing the effects of physical therapy on balance and postural control after stroke. Methods Six databases were searched until January 2019. Two independent reviewers selected randomised trials, extracted data, and assessed risk of bias. We conducted subgroup meta-analyses according to the language of study publication, then compared TEE between SPEL and SPNEL subgroups by using a random-effects meta-regression model. Results From 13,123 records, 132 SPEL (n = 5219) and 13 SPNEL (n = 693) were included. SPNEL had a weight in the pooled estimate (8.2%) significantly lower than SPEL (91.8%; p<0.001). Compared to SPEL, SPNEL had both significantly worse methodological quality (p = 0.002) and quality of reporting for blinding of outcome assessment (p<0.001); and a significantly worse quality of reporting for incomplete outcome data (p<0.001). SPNEL had a significantly worse precision (i.e. inverse of variance) of TEE than SPEL (p = 0.005). Overall, the TEE was not significantly different between SPNEL and SPEL (standardised mean difference -0.16, 95% confidence interval [-0.53; 0.22], heterogeneity I2 = 78%). However, when PT was compared to sham treatment or usual care, SPNEL significantly over-estimated treatment effects (SMD -0.68, 95%CI [-1.03; -0.33], I2 = 39%) compared to SPEL. Restriction of the search to SPEL only did not change the direction of TEE for 8 out of 9 comparisons. Conclusions SPNEL had a worse methodological quality than SPEL and were likely to over-estimate treatment effect. If inclusion of SPNEL in a systematic review is considered to be relevant, the impact of such studies on TEE should be explored by sensitivity analyses to ensure the findings validity.