Establishing gold standards for System-Level Measures: A modified Delphi consensus process

Abstract

Objective: To establish aspirational ‘gold standards’ for a suite of System-Level Measures (SLMs) being used by Counties Manukau Health (CM Health), a New Zealand (NZ) District Health Board. Design: This study employed a multi-stage, multi-method modified Delphi consensus process. Setting: The Delphi consensus process involved virtual (email) communication between participants (Round 1) and a structured face-to-face meeting (Round 2) held in Auckland, NZ. Participants: Participants comprised of health professionals, managers, academics and quality improvement experts with an interest in the use of SLMs. Interventions: Participants in the first round received a letter requesting their participation in an anonymous Delphi. The second round involved national and international health system experts taking part in a structured, facilitated face-to-face meeting. Participants reviewed 15 SLMs in total. The SLMs all related to the three domains of the Triple Aim: Population Health, e.g. life expectancy at birth; Patient Experience of Care, e.g. rate of adverse events; and Cost and Productivity, e.g. healthcare expenditure per capita. Main Outcome Measures: For a proposed gold standard to be agreed and established for each SLM. Results: Twelve participants took part in Round 1, with 19 participating in Round 2. The process established agreement on a gold standard for each of the 15 reviewed SLMs. Conclusion: We demonstrated that the Delphi consensus process can be used to establish gold standards for a suite of SLMs used by a NZ Health Board (CM Health).