A randomized controlled trial of league tables and control charts as aids to health service decision-making

Abstract

Objectives. Health service managers and other decision-makers are required to act on the basis of data. Little attention has been paid to the effects of data presentation on the decisions taken. This study uses a randomized controlled trial design to investigate the effects of two forms of data presentation - league tables and control charts - on health service decision-makers. Methods. Directors of public health in 122 health authorities in the UK were mailed three case studies and a questionnaire. The case studies showed data on variations in mortality by health service provider. The questionnaire asked them to indicate whether they would take action as a result of the data and to identify the health service providers on whom they would take action. Participants were randomly allocated to receive the same data in the form of ranked histograms (league tables) or control charts. Main outcome measure. The percentage of participants who would take action on health service providers. Results. Fifty-seven questionnaires were returned. For each case study, respondents receiving data as league tables stated they would take action on significantly more health service providers than those receiving data as control charts: for the first case study, the percentages were 3.3% versus 1.8% (P < 0.001) for league tables and control charts, respectively; for the second case study, 15.9% versus 6.7% (P = 0.029), respectively; and for the third, 5.9% versus 0.7% (P = 0.002), respectively. Respondents receiving data as league tables were significantly more likely to request further information on case mix. Conclusions. Compared with league tables, health service decision-makers identify fewer outliers for further action when performance data are presented as control charts. They also reduce the tendency to request further information. Using control charts rather than league tables for the routine presentation of comparative data would reduce over-investigation of unusual performance. © International Society for Quality in Health Care and Oxford University Press 2004; all rights reserved.