ISQUA16-2572AN INTEGRATED TOOL FOR EVALUATING THE FRENCH MANDATORY QUALITY INDICATORS USED FOR HOSPITAL IMPROVEMENT, PUBLIC DISCLOSURE AND REGULATION

Abstract

Objectives: Since ten years, quality indicators (QI) are mandatorily measured in all French hospitals. They are used for hospital quality improvement, public disclosure and regulation goals. After a decade of use, the Ministry of Health and the French National Authority for Health set up a task force to evaluate these QI. The goal was to provide a decision aid to the Ministry and HAS in terms of withdrawing, revising or continuing their use. As there was no available integrated tool, the task force started with the development and test of a tool during the year 2015. This tool aimed at evaluating the performance (measurement feasibility, metrological performance and relevance for users) of QIs according to each of the three goals. Method(s): From a literature review and a review of national initiatives, we identified potential criteria for assessment. Using a modified Rand/UCLA Appropriateness Method, the appropriateness of these criteria was checked for evaluating the indicators according to each of the three goals. The taskforce, constituted by quality indicators researchers, hospital managers, quality managers and clinicians, extracted a list of criteria for each goal. Each criterion was assessed using a quantitative approach or a qualitative approach: The indicator's results collected the previous years were analyzed; the scientific soundness of the indicator development was reviewed; field hospital workers (doctors, nurses, managers), health authorities employees and patients were interviewed. All these data results were discussed by the task force, summarized and displayed on spider charts presenting standardized scores. This integrated tool was tested on four national process QIs related to nosocomial infections management. Result(s): Three major experiences were studied, the Qualify method from the BQS institute in Germany, the Dutch Appraisal of Indicators through Research and Evaluation (AIRE) method and the method developed for AHRQ by the UCSF-Stanford Evidence-based Practice Center to refine the original set of HCUP QIs. Among the fourteen retrieved potential criteria, 12 were selected as appropriate for the evaluation of QIs for regulation use: Relevance (Importance of the quality characteristic captured for the health care system; Benefit/ability to take decision based on the results; Potential risks/Side effects), Feasibility (Understandability and Interpretability of the indicator and its results; Barriers to Implementation due to data collection effort; Delays related to data production), scientific soundness of the indicator development (Indicator evidence; Validity; Risk adjustment) and current metro-logical performance (Indicator Expression can be Influenced by Providers; Discriminatory power; Sensitivity to change). Among these, 11 were selected for hospital improvement and 7 for public disclosure. Applied to the four QIs, the task force proposed to withdraw the indicator related to multidrug-resistant bacteria management and to undertake major revision on the three others. Conclusion(s): This integrated tool proved to be operational on four QIs. More research is needed to finalize the tool on other types of indicators. A core set of seven assessment criteria was identified to evaluate QIs for hospital improvement, public disclosure and regulation. Additional five criteria are needed for the goal of hospital improvement and regulation. These results have been presented to the Ministry QIs advisory group.