PMA9: BAYESIAN OR CLASSICAL DESIGN AND ANALYSIS: DOES IT MAKE A DIFFERENCE??

Abstract

INTRODUCTION: The utility of research results is measured primarily by its effects on decisions. Underpinning research are methods appropriate to the question or hypothesis. The role of Classical and Bayesian approaches remains in dispute in health services research. The goal of this study was to determine if results differ when both analytic techniques are used with the same dataset. METHOD: We searched MEDLINE and related databases for English-language articles published 1 January 1978 through 31 August 1999. We combined Bayesian and Classical statistics search terms, and their variants, with randomized control trials (RCTs) and meta analyses. RESULTS: Searches found 18 studies in 14 publications that met all criteria for review?9 RCTs, 8 meta-analyses and 1 epidemiological estimate. Statistical analyses using both methods agreed in 5 RCTs, 4 meta-analyses, and for the epidemiological estimates. For 4 RCTs where results disagreed, Classical analysis the experimental intervention was efficacious compared to the control and Bayesian reanalysis concluded the experimental intervention was not proven efficacious. Classical meta-analyses of the four studies where results disagreed concluded the experimental intervention was not better than the control; Bayesian reanalysis concluded the intervention was efficacious. CONCLUSION: The conventional wisdom that Classical and Bayesian methods will give similar answers is not supported by this study. Disagreement on many fundamental beliefs between Classical and Bayesian statistics means continuing debate. One way to resolve this debate is for proponents of each technique to decide together the circumstances for use of each method and analytic framework. If the experts do not agree on the methodological requirements, other decision-makers likely will force their own views, driven mainly by other pressures like cost control.