On the appropriate interpretation of evidence: The example of culture media and birth weight

Abstract

In the specific example discussed here, a careful evaluation of the trial would have to tentatively conclude that there is sufficient evidence to suggest that the use of these two different culture media systems may lead to differences in birth weight. The mechanism is unknown, but must be directly related to either the media composition or ancillary changes necessitated by the use of these media in accordance with recommended practice. An 'unlucky' chance finding is the only alternative explanation. Therefore, until there is compelling direct evidence to the contrary, we have to take the possibility of differential effects of culture conditions on neonatal and long-term health seriously and observe due vigilance. We would reiterate the call for more RCTs and for the routine collection of culture media use and formulation. In conclusion, the debate about this potentially important finding has illustrated the need for careful interpretation of trial results and exemplified errors in interpretation which are not uncommon elsewhere. Most of these relate to over-reliance on P-values and failure to properly consider effect sizes and confidence intervals (Farland et al., 2016). Although the statistical principles underpinning this have been popularized and disseminated for several decades (Gardner and Altman, 1988) it is clear as a community we still have much to learn.