Randomised trials of new procedures: problems and pitfalls.

Abstract

Randomised trials that compare new procedures with established ones must avoid prerandomisation bias and must allocate patients to treatment groups based on objective or quantitative criteria, not on subjective clinical judgment. Risk, length of follow up, and sample size must be used to calculate the statistical power of the study, so that a significant difference between treatments does not remain undetected (a type II error). There should already be sufficient experience with the new procedure so that complication rates have stabilised, and participating operators are equally comfortable with all procedures being studied. Even with the above stipulations, randomised trials that compare medical with procedural treatment pose additional problems (many of which have been omitted from this necessarily brief discussion); few such studies have had a major impact on clinical practice. The most useful randomised studies of procedures are those that compare one procedure with another, or those that assess a specific refinement in an established procedure, such as the use of different anticoagulation regimens for coronary stents. Fortunately, clinically useful information has always been available from non-randomised studies. The recent trend towards meta-analysis of large clinical series can substitute for those randomised studies that are unlikely to be helpful.