TOPIC PAPER Rating the quality of evidence and the strength of recommendations using GRADE Steven E. Canfield ��� Philipp Dahm Received: 9 November 2010 / Accepted: 22 February 2011 / Published online: 9 March 2011 �� Springer-Verlag 2011 Abstract Objectives Urologists can benefit from a standardized system for guideline development and presentation. This article introduces the GRADE system and explains how it may be useful for Urologic physicians, in their practice and in their healthcare systems. Methods The GRADE system is reviewed. Specific aspects of how GRADE rates the quality of the evidence and the strength of recommendations are explored. Results GRADE can provide explicit and structured guidance, which separates the quality of evidence from the strength of recommendations. This information can be used by consumers of guidelines, including patients, physicians, and policy makers. Conclusions Urologists can benefit from a more trans- parent and rigorous framework when formulating recom- mendations. GRADE is an emergent proposal with broader implications for healthcare policy as well. Keywords Guidelines Levels of evidence Urology Introduction Urologic guidelines can be difficult to compare due to wide variations in how they rate the quality of evidence and present their recommendations. Often, the authors cannot offer an actual recommendation due to perceived gaps in the evidence, or when a recommendation is offered, it may be unclear what the ranking of the recommendation implies. Unfortunately, such guidelines may not provide much actual guidance and may introduce confusion. Grading of Recommendations Assessment, Development, and Evaluation (GRADE) is a system which provides clear and concise information on both the quality of the evidence and the strength of the recommendation. The system can be used when developing systematic reviews and when for- mulating recommendations in the context of guidelines. Information on patient important outcomes is presented in a systematic and explicit fashion which can be used by physicians, patients, and policy makers. Why is a better system needed? The purpose of a guideline is to provide a summary of best- known practices for a given topic, which can aid practitio- ners, may lead to improved outcomes for patients, and may inform healthcare policy. Currently, there is a confusing array of urological guideline formats which can be demon- strated by a brief assessment of examples from three major organizations (Table 1). Each of these organizations is internationally respected and performs high-quality evi- dence-based evaluations for their guideline recommenda- tions. The Scottish Intercollegiate Guideline Network (SIGN) denotes the level of evidence as A, B, or C and provides a recommendation [1]. As an example, consider recommendations for improving urinary control after radical prostatectomy. The SIGN guideline recommends that pelvic floor muscle exercise should be considered, grade B [2]. The National Comprehensive Cancer Network (NCCN) devel- ops algorithms which attempt to identify logical disease S. E. Canfield (&) Division of Urology, UT Medical School at Houston, 6431 Fannin St. MSB 6.018, Houston, TX 77030, USA e-mail: steven.canfield@uth.tmc.edu P. Dahm Department of Urology, College of Medicine, University of Florida, Gainesville, FL, USA 123 World J Urol (2011) 29:311���317 DOI 10.1007/s00345-011-0667-2

pathways and guideline elements (recommendations) [3]. The NCCN suggests that preserving urethral length and avoiding damage to the external sphincter can ������reduce������ incontinence, and preserving the bladder neck may ������decrease the risk������ of incontinence, category 2A [4]. The European Association of Urology (EAU) presents both levels of evidence and grades for recommendations derived from the Agency for Healthcare Research and Quality (AHRQ) [5]. They recommend that ������some preoperative or immediate postoperative instructions in pelvic muscle training for men undergoing radical prostatectomy may be helpful������, grade B [6]. These three examples illustrate the diverse approach undertaken by just a few of the many important organiza- tions which seek to provide practical guidance to urolo- gists. Yet an attempt to compare the information provided by these different groups would prove difficult. The methodology used is too different, and there is no cross- walk from guidelines of one organization to another. In many cases, it also remains unclear what the considerations were how guideline developers arrived from a given level of evidence to a certain recommendation. In other cases, no specific recommendation is made. A structured format for guideline development and presentation which rates the quality of available evidence and defines the factors involved when grading the strength of the recommenda- tions would be a powerful tool to synthesize information and could provide better guidance to practitioners. What is GRADE and how was it developed? The GRADE system was created in response to the need for a more unified and transparent approach to guidelines creation and reporting [7]. Individuals from all over the world, many from leading organizations involved in defining levels of evidence, including NICE, ARHQ, and the National Health and Medical Research Council (NHMRC) formed the GRADE working group in 2000 and have since been working in the development of the GRADE system [8]. This framework has now been adopted as the standard for guideline development by over 50 international organizations, including the World Health Organization (WHO), the Cochrane Collaboration, SIGN, AHRQ, and the Centers for Disease Control and Prevention (CDC). Current resources on the GRADE methodology include the original series for guideline developers pub- lished in the British Medical Journal [7���11] and the GRADE working group website [12]. How does GRADE actually work? Just as in any well-conducted research study, a guideline should employ well-designed clinical questions which contain the four components known as ������PICO������: patient, intervention, comparison, and outcome of interest [13]. For example, consider a guideline which is being developed for ureteral calculi. One issue to be addressed within this guideline is the role of medical expulsive therapy. A well- designed question might ask ������In adult patients with ure- teral calculi, does medical expulsive therapy compared to standard care improve outcomes?������ But what is the ������out- come?������ The GRADE system suggests attempting to iden- tify all potentially relevant outcomes for each specific question and rate their relative importance a priori. In this example, beneficial outcomes could include stone passage rates, reduced pain, fewer complications, and fewer related surgeries. Societal outcomes could include reduced resource utilization. Physiologic outcomes could include relaxation of the ureter. Negative outcomes of the added medication could include hypotension, edema, erectile dysfunction, and increased cost. To add clarity to such a long list, GRADE ranks the relative importance of out- comes to clinical decision making on a scale of 1 (not important) to 9 (critical) [9] (Fig. 1). For all critical and most important outcomes, guideline panels using GRADE then review the available quality of evidence for each Table 1 Examples of current guideline rating and grading formats SIGN NCCN EAUa A���randomized controlled trials (RCTs) or systematic reviews of RCTs 1���high-quality evidence along with uniform level of consensus A���clinical studies of good quality and consistency addressing the specific recommendations and including at least one randomized controlled trial B���non-randomized trials and other observational studies 2a and 2b���lower-quality evidence with uniform or non-uniform consensus B���well-conducted clinical studies, but without randomized clinical trials C���expert opinion 3���any quality evidence but with major disagreement among panelists C���recommendations made despite the absence of directly applicable clinical studies of good quality a EAU levels of evidence applied as well based on Oxford centre for evidence-based medicine levels of evidence 312 World J Urol (2011) 29:311���317 123