Interpretation of Statistical Significance - Exploratory Versus Confirmative Testing in Clinical Trials, Epidemiological Studies, Meta-Analyses and Toxicological Screening (Using Ginkgo biloba as an Example)

Abstract

The terms "significant" and "p-value" are important for biomedical researchers and readers of biomedical papers including pharmacologists. No other statistical result is misinterpreted as often as p-values. In this paper the issue of exploratory versus confirmative testing is discussed in general. A significant p-value sometimes leads to a precise hypothesis (exploratory testing), sometimes it is interpret as "statistical proof" (confirmative testing). A p-value may only be interpreted as confirmative, if (1) the hypothesis and the level of significance were established a priori and (2) an adjustment for multiple testing was carried out if more than one test was performed. Screening programmes (e.g. the U.S. National Toxicology Programme on Ginkgo biloba) are typical for exploratory results. Controlled randomised trials include typically one confirmative test of the primary outcome variable and several exploratory tests of secondary outcome variables, as well as exploratory subgroup analyses. Some studies deliver p-values, which are more meaningful than merely exploratory, whilst other p-values appear to be more or less confirmative. Epidemiological studies and meta-analyses may lead to p-values, which are somewhat between exploratory and confirmative. We propose to consider exploratory and confirmative as a bipolar continuum. Nevertheless, authors of a study protocol are advised to design their study in a clearly exploratory or strictly confirmative manner. We furthermore recommend that each published significant p-value is explicitly denoted as exploratory or confirmative in addition to the appropriate descriptive results.