Development of appropriateness criteria for colonoscopy: Comparison between a standardized expert panel and an evidence-based medicine approach

Abstract

Objectives. To assess the degree of agreement between appropriateness criteria for the use of colonoscopy developed by a standardized expert panel method and evidence from published studies. Design. Descriptive, agreement study. Setting. Multidisciplinary panel; primary care practice in Switzerland. Participants. Nine national experts; 577 primary care patients referred for colonoscopy; 154 published papers. Interventions. Evaluation of the appropriateness of 402 possible clinical indications for colonoscopy, based on a comprehensive review of the literature. Main outcomes measures. Proportion of agreement (weighted kappa), between panel- and literature-based appropriateness categories (appropriate, uncertain, inappropriate) for theoretical and actual indications encountered. Results. Nineteen of 402 indications rated by the panel could be based on the evidence retrieved from eight randomized clinical trials. A 68% agreement (kappa = 0.52) was found between panel- and study-based criteria. The addition of an uncontrolled trial and seven observational studies yielded a 71% agreement (kappa = 0.63). Agreement was similar when examining 577 actual cases: 69% agreement, kappa = 0.47. Agreement between panel-based indications and published evidence was not influenced by the perceived comprehensiveness and the apparent quality of the published reports. Conclusions. Evidence for the appropriateness of most indications for colonoscopy could not be derived directly from the published literature. Agreement between appropriateness criteria developed by an expert panel and evidence from published studies was moderate to good, where available. New approaches should be sought in order to systematically integrate complementary evidence obtained from clinical trials and expert panels into practice guidelines.