The use and limitations of the fragility index in the interpretation of clinical trial findings

Abstract

The fragility index (FI) has been recommended for use as an additional statistic when presenting the results of randomized controlled trials (RCTs). The FI in a completed RCT is the smallest number of subjects whose status needs to be changed, such as from nonresponder to responder, for a statistically significant finding to lose its statistical significance. A small FI suggests that a finding is fragile; a large FI suggests that the finding is robust. Whereas an FI value of 0-1 indicates extreme fragility, there is no cutoff to separate what is small and what is large for the FI. The FI is useful because it helps readers understand significant findings of an RCT in a different and more intuitive way. The FI has limitations. It can only be calculated in the context of an RCT, and only when binary outcomes are compared between 2 groups. It should not be calculated in nonrandomized studies, because it cannot be adjusted for the biasing effect of confounding variables, nor in time-to-event studies, because it cannot include the effect of time. Interpretation of the FI can be problematic when the number of subjects who drop out for unknown reasons is large. RCTs with small samples and RCTs in which the event of interest is rare tend to be fragile. However, the most important limitation of the FI is that it revolves around the much decried use of a statistical threshold (usually P < .05) for determining the significance of a study finding. At best, the FI complements the understanding of the results of an RCT with statistically significant findings for categorical outcomes. It should be used and interpreted in the context of other statistical information, including summary statistics, measures of effect size, and confidence intervals.