III validity review of performance measures

Abstract

A structured process for evaluating the validity of performance measures is essential if the rates generated by these measures are to be relied upon for large-scale programmes to improve clinical care. In this evaluation process a panel of objective clinicians who were not involved in developing the performance measures examines the criteria, coding instructions, and coding form. The panel's job is to determine the degree to which a sample of patient records containing treatment elements that have been flagged for nonconformance to certain criteria are truly nonconformant. Modification of the performance measure according to the recommendations of the panel provides an important means of reducing measurement error by eliminating obvious sources of false positive findings. This article describes the procedures followed and the materials used to validate the performance measures developed for the project to Develop and Evaluate Methods for Promoting Ambulatory Care Quality (DEMPAQ), a research project contracted by the Health Care Financing Administration of the United States government [1-3]. DEMPAQ is described in Fig. 1. In the DEMPAQ project, record review is conducted by entering data directly from patient ambulatory records into a computer system. Instructions for abstracting the data are displayed on the computer monitor.