Validation and reliability of a guideline appraisal mini-checklist for daily practice use

Abstract

Background: The use of comprehensive instruments for guideline appraisal is time-consuming and requires highly qualified personnel. Since practicing physicians are generally busy, the rapid-assessment Mini-Checklist (MiChe) tool was developed to help them evaluate the quality and utility of guidelines quickly. The aim of this study was to validate the MiChe in comparison to the AGREE II instrument and to determine its reliability as a tool for guideline appraisal. Methods: Ten guidelines that are relevant to general practice and had been evaluated by 2 independent reviewers using AGREE II were assessed by 12 GPS using the MiChe. The strength of the correlation between average MiChe ratings and AGREE II total scores was estimated using Pearson's correlation coefficient. Inter-rater reliability for MiChe overall quality ratings and endorsements was determined using intra-class correlations (ICC) and Kendall's W for ordinal recommendations. To determine the GPS' satisfaction with the MiChe, mean scores for the ratings on five questions were computed using a six-point Likert scale. Results: The study showed a high level of agreement between MiChe and AGREE II in the quality rating of guidelines (Pearson's r = 0.872; P < 0.001). Inter-rater-reliability for overall MiChe ratings (ICC = 0.755; P < 0.001) and endorsements (Kendall's W = 0.73; P < 0.001) were high. The mean time required for guideline assessment was less than 15 min und user satisfaction was generally high. Conclusions: The MiChe performed well in comparison to AGREE II and is suitable for the rapid evaluation of guideline quality and utility in practice. Trial registration: German Clinical Trials Register: DRKS00007480