Concerns regarding the standard deviation of individual responses for assessing treatment response heterogeneity

Abstract

The estimation of treatment response heterogeneity (TRH) is increasingly important as medicine moves toward personalized approaches. Although various statistical methods have been proposed to quantify TRH in parallel-group trials, the standard deviation of individual responses (SDIR) has gained prominence within physiological research. This method is intended to quantify individual response variation by comparing standard deviations of change scores between intervention and control groups. We acknowledge that SDIR represents an improvement over many other flawed approaches that often involve responder counting. However, SDIR has critical limitations: 1) it cannot overcome the fundamental problem of causal inference because the correlation between potential outcomes remains unidentifiable, 2) it is incorrectly predicated on the assumption that TRH is present only when treatment group variance exceeds control group variance, and 3) it is statistically inefficient. We present an alternative framework, which involves assessing heteroskedasticity and estimating the bounds for the standard deviation of treatment effects (SDD). The presence of heteroskedasticity between treatment groups is a sufficient but not necessary condition for the presence of TRH. Furthermore, SDD makes fewer assumptions than SDIR and, therefore, paints a more complete picture of potential TRH. Using data from a published exercise physiology study, we demonstrate how SDD can better characterize uncertainty in TRH estimation. We recommend that researchers probe TRH by assessing heteroskedasticity, providing bounds for SDD, and estimating outcome distributions and probabilities while carefully crafting the theoretical rationale for the presence of TRH.