SYSTEMATIC REVIEW Open Access Computerized clinical decision support systems for chronic disease management: A decision- maker-researcher partnership systematic review Pavel S Roshanov1, Shikha Misra2, Hertzel C Gerstein3,4, Amit X Garg5, Rolf J Sebaldt3, Jean A Mackay6, Lorraine Weise-Kelly6, Tamara Navarro6, Nancy L Wilczynski6 and R Brian Haynes3,4,6*, for the CCDSS Systematic Review Team Abstract Background: The use of computerized clinical decision support systems (CCDSSs) may improve chronic disease management, which requires recurrent visits to multiple health professionals, ongoing disease and treatment monitoring, and patient behavior modification. The objective of this review was to determine if CCDSSs improve the processes of chronic care (such as diagnosis, treatment, and monitoring of disease) and associated patient outcomes (such as effects on biomarkers and clinical exacerbations). Methods: We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid���s EBM Reviews database, Inspec, and reference lists for potentially eligible articles published up to January 2010. We included randomized controlled trials that compared the use of CCDSSs to usual practice or non- CCDSS controls. Trials were eligible if at least one component of the CCDSS was designed to support chronic disease management. We considered studies ���positive��� if they showed a statistically significant improvement in at least 50% of relevant outcomes. Results: Of 55 included trials, 87% (n = 48) measured system impact on the process of care and 52% (n = 25) of those demonstrated statistically significant improvements. Sixty-five percent (36/55) of trials measured impact on, typically, non-major (surrogate) patient outcomes, and 31% (n = 11) of those demonstrated benefits. Factors of interest to decision makers, such as cost, user satisfaction, system interface and feature sets, unique design and deployment characteristics, and effects on user workflow were rarely investigated or reported. Conclusions: A small majority (just over half) of CCDSSs improved care processes in chronic disease management and some improved patient health. Policy makers, healthcare administrators, and practitioners should be aware that the evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies measuring patient outcomes. Background Chronic conditions present patients, practitioners, and healthcare systems with some unique demands, includ- ing recurrent visits, adherence to complex care plans, long-term disease and treatment monitoring, behavior modification, and patient self-management. For the many patients with multiple co-morbidities [1], overlapping or diverging care plans may further compli- cate these processes. Computerized clinical decision support systems (CCDSSs) may help practitioners meet the requirements of chronic care. These systems analyze a patient���s char- acteristics to provide tailored recommendations for diag- nosis, treatment, patient education, adequate follow-up, and timely monitoring of disease indicators. For exam- ple, Holbrook et al. [2,3] gave providers and diabetic patients access to a web-based system that offered care advice, allowed monitoring of diabetes risk factors, and * Correspondence: bhaynes@mcmaster.ca 3Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, ON, Canada Full list of author information is available at the end of the article Roshanov et al. Implementation Science 2011, 6:92 http://www.implementationscience.com/content/6/1/92 Implementation Science �� 2011 Roshanov et al licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

tracked key care targets. As with any health interven- tion, however, rigorous testing is warranted to deter- mine whether CCDSSs improve chronic care processes and patient outcomes. In our previous review of the effects of CCDSSs [4], we analyzed 100 randomized and non-randomized stu- dies published until September 2004, 40 of which assessed the effects of CCDSSs on disease management. Of these 40 studies, 37 measured processes of care of which 62% (23) showed an improvement, and 27 mea- sured patient outcomes of which 19% (5) showed an improvement. The quality of the studies varied widely, but improved over time. Many new randomized controlled trials (RCTs) have been published in this field since our previous work, potentially documenting important advances. Recogniz- ing that the management of chronic disease has unique characteristics, we wished to review the impact of CCDSSs on the quality and effectiveness of chronic care. We had the opportunity to include the perspectives of senior hospital managers and front-line healthcare practi- tioners to ensure that relevant data were extracted and summarized���a level of stakeholder engagement that has not been included in other reviews [5-8]. Methods We previously published the details of our review proto- col, openly accessible at http://www.implementa- tionscience.com/content/5/1/12[9]. These methods are briefly summarized here, along with details specific to this review of CCDSSs for chronic disease management. Research question Do CCDSSs improve chronic disease management pro- cesses or patient outcomes? Partnering with decision makers We conducted this review in partnership with indivi- duals responsible for implementing CCDSSs in our region [9]. Decision makers, both managers and clini- cians, met with the review team periodically to discuss direction and specific details for the data extraction, analysis, presentation and interpretation of results. Search strategy Full details of our search strategy are in our review proto- col [9]. In summary, we searched MEDLINE, EMBASE, Ovid���s Evidence-Based Medicine Reviews, and Inspec until 6 January 2010, and reviewed the reference lists of included RCTs and relevant systematic reviews. We screened articles for eligibility in two stages: a duplicate, independent review of titles and abstracts followed by a duplicate, independent, full-text review of potentially eligi- ble articles, with a third reviewer resolving disagreements. Study selection We selected RCTs of a CCDSS used by a health care provider for management of chronic conditions, pub- lished up to 6 January 2010 in any language that mea- sured CCDSS impact on processes of care or patient outcomes. We included RCTs in any language that com- pared patient care with a CCDSS to routine care with- out a CCDSS and evaluated clinical performance (i.e., a measure of process of care) or a patient outcome. Addi- tionally, to be included in the review, the CCDSS had to provide patient-specific advice that was reviewed by a healthcare practitioner before any clinical action. Studies were excluded if the system was used solely by students, only provided summaries of patient information, pro- vided feedback on groups of patients without individual assessment, only provided computer-aided instruction, or was used for image analysis. Trials included in our previous review [4] were included if they were eligible. Trials of CCDSSs for managing narrow therapeutic index medications used in some chronic conditions (such as warfarin in atrial fibrillation [10]) were not included in this review, but are discussed in our review for therapeutic drug monitoring and dosing. Data extraction To meet the needs of our management and clinical part- ners, we extracted study characteristics (e.g., study design, size, setting, authorship, funding, and year of publication) and system characteristics (e.g., integration with other systems, user interface elements, methods of data entry and delivery of recommendations, target users, and implementation details such as pilot testing and user training). Disagreements were resolved by a third reviewer or by consensus. We contacted primary authors to provide missing data and to assess the accu- racy of the extracted data 78% (43/55) provided input. For the remaining trials, a trained reviewer assessed the extraction form against the full-text to confirm accuracy. Assessment of study quality Using a 10-point scale, pairs of reviewers independently evaluated the selected trials on five dimensions of qual- ity, including concealment of allocation, appropriate unit of allocation, appropriate adjustment for baseline differences, appropriate blinding of assessment, and ade- quate follow-up [9]. We used a 2-tailed Mann-Whitney U test to compare methodologic scores between trials published before the year 2000 and those published later to determine if trial quality has improved with time. Assessment of CCDSS intervention effects We assessed the effectiveness of CCDSSs in each trial for improving process of care and patient outcomes. We defined process outcomes as changes in care activities Roshanov et al. Implementation Science 2011, 6:92 http://www.implementationscience.com/content/6/1/92 Page 2 of 16

such as diagnosis, treatment, and monitoring of disease. Examples of patient outcomes included changes in blood pressure, clinical events and health-related quality of life. We judged a CCDSS effective if it produced a statistically significant (p 0.05) improvement in a pri- mary chronic disease outcome or in ���50% of multiple relevant pre-specified outcomes. We considered primary any outcome that trial reports described as ���primary��� or ���main.��� If authors did not designate a primary outcome, we considered the outcome used to calculate the trial���s sample size to be primary, if reported. When there were no pre-specified outcomes, the system was considered effective if it produced an improvement in ���50% of all reported chronic disease outcomes. Our assessment cri- teria are more specific than those used in our 2005 review [4] therefore, the assignment of effect was adjusted for some trials included in the review. Data synthesis and analysis We summarized data using proportions, medians, and ranges. Denominators vary in some proportions because not all trials reported relevant information. All analyses were carried out using SPSS, version 15.0. We did not attempt a meta-analysis because of study-level differ- ences in participants, clinical settings, disease conditions, interventions, and outcomes measured. We conducted a sensitivity analysis to assess the pos- sibility of biased results in studies with a mismatch between the unit of allocation (e.g., clinicians) and the unit of analysis (e.g., individual patients without adjust- ment for clustering). We compared success rates between studies with matched and mismatched analyses using chi-square for comparisons. No differences in reported success were found for either process of care outcomes (Pearson X2 = 1.41, p = 0.24) or patient out- comes (Pearson X2 = 1.45, p = 0.23). Accordingly, results have been reported without distinction for mismatch. Results Figure 1 shows a flow diagram of included and excluded trials. We identified 166 trials of CCDSSs and Cohen���s for reviewer agreement on trial eligibility was 0.93 (95% confidence interval [CI], 0.91 to 0.94). In this review, we included 71 publications describing 55 trials (33% of total) about management of chronic diseases [2,3,11-79]. Thirty-eight included studies contributed outcomes to both this review and other CCDSS interventions in the series three studies [30,53,62] to four reviews, 12 studies [21,25,28,31-33,42-44,51,52,54,55,57-61,74] to three reviews, and 23 studies [2,3,11,12,18,19,23,27, 35-38,40,41,45,46,48-50,56,67,71-73,77,79] to two reviews but we focused here on outcomes relevant to the management of chronic disease. Summary of trial quality is reported in Additional file 1, Table S1 system characteristics in Additional file 2, Table S2 study characteristics in Additional file 3, Table S3 outcome data in Table 1 and Additional file 4, Table S4 and other CCDSS-related outcomes in Additional file 5, Table S5. Study quality Additional file 1, Table S1 presents details of our meth- odological quality assessment. Of the 55 trials, 53% reported adequate concealment of allocation [2,3,13,18,20,27,29,31-34,37,39,40,47-58,60-67,72-75] 78% showed no differences in baseline characteristics between study groups or adjusted accordingly [2,3,11-13,18-21,23-25,28,29,34,36,38-58,60-67,70-72,74- 76,78,79] 53% allocated entire wards or practices to each study group [11,12,14-18,25,28-35,37,39,46- 49,51,54-59,62-64,67,70,73,76-79] all except one used objective outcomes or blinding of outcome assessments [23] and 60% achieved a ���90% follow-up rate for their unit of analysis [11-13,18-24,27,30,35,36,39,40,46, 47,50-55,59-62,65-70,73,74,76,77]. The overall quality of trials was good (median methods score, 8 ranging from 2 to 10) and improved with time (median methods score before versus after year 2000, 7 versus 8, 2-tailed Mann-Whitney U = 137 p = 0.005), possibly because early trials often failed to conceal allocation or to achieve adequate follow-up. Records identified through database searching (n = 14,794) Screening Included Eligibility Identification Additional records identified from previous review (n = 86) and through other sources (n = 72) Records after duplicates removed (n = 14,188) Records screened (n = 14,188) Records excluded (n = 13,859) Full-text articles assessed for eligibility (n = 329) Full-text articles excluded, with reasons (n = 163) 74 Not RCTs 50 Did not evaluate CCDSS 14 Supplemental reports 9 Severe methodological flaws 7 Did not meet CCDSS definition 4 Did not report outcomes of interest 4 Only abstract published 1 Included in previous review Studies included in review series (n = 166) Studies included in this review (met chronic disease management criteria) (n = 55) Figure 1 Flow diagram of included and excluded studies for the update 1 January 2004 to 6 January 2010 with specifics for chronic disease management*. * Details provided in: Haynes RB et al. [9] Two updating searches were performed, for 2004 to 2009 and to 6 January 2010 and the results of the search process are consolidated here. Roshanov et al. Implementation Science 2011, 6:92 http://www.implementationscience.com/content/6/1/92 Page 3 of 16