Probability, proof, and clinical significance

Abstract

A study reports that treatment A "provided signifi cantly better pain relief" than treatment B. How do you know if the effect is real or due to chance? Assuming that the difference between treatment groups on the outcome is statistically signifi cant, does this mean that your patients will have a clinically signifi cant improvement? This article explores these questions as well as provides some additional points to consider when critically appraising and conducting research. When a difference in an outcome (eg, pain) between exposures (eg, treatment groups) is observed, one needs to consider whether the effect is truly due to the exposure or if alternate explanations are possible. In other words, in order to evaluate the validity of a research study, factors that might distort the true association and/or infl uence its interpretation need to be carefully considered. This means evaluating the role of bias and considering the study's statistical precision. Bias relates to systematic sources of error which need to be considered. (Bias will be discussed in more detail in future issues). By contrast, evaluation of statistical precision involves consideration of random error within the study, random error being the part of the study that cannot be predicted, ie, that part attributable to chance. In addition, one needs to consider whether a clinically meaningful improvement is represented.