On the bias of complete- and shifting-case meta-regressions with missing covariates

Abstract

Missing covariates is a common issue when fitting meta-regression models. Standard practice for handling missing covariates tends to involve one of two approaches. In a complete-case analysis, effect sizes for which relevant covariates are missing are omitted from model estimation. Alternatively, researchers have employed the so-called "shifting units of analysis" wherein complete-case analyses are conducted on only certain subsets of relevant covariates. In this article, we clarify conditions under which these approaches generate unbiased estimates of regression coefficients. We find that unbiased estimates are possible when the probability of observing a covariate is completely independent of effect sizes. When that does not hold, regression coefficient estimates may be biased. We study the potential magnitude of that bias assuming a log-linear model of missingness and find that the bias can be substantial, as large as Cohen's d = 0.4–0.8 depending on the missingness mechanism.