Percentile-based empirical distribution function estimates for performance evaluation of healthcare providers

Abstract

Hierarchical models are widely used to characterize the performance of individual healthcare providers. However, little attention has been devoted to systemwide performance evaluations, the goals of which include identifying extreme (e.g. the top 10%) provider performance and developing statistical benchmarks to define high quality care. Obtaining optimal estimates of these quantities requires estimating the empirical distribution function (EDF) of provider-specific parameters that generate the data set under consideration. However, the difficulty of obtaining uncertainty bounds for a squared error loss minimizing EDF estimate has hindered its use in systemwide performance evaluations. We therefore develop and study a percentile-based EDF estimate for univariate provider-specific parameters. We compute order statistics of samples drawn from the posterior distribution of provider-specific parameters to obtain relevant assessments of uncertainty of an EDF estimate and its features, such as thresholds and percentiles. We apply our method to data from the Medicare end stage renal disease programme, which is a health insurance programme for people with irreversible kidney failure. We highlight the risk of misclassifying providers as exceptionally good or poor performers when uncertainty in statistical benchmark estimates is ignored. Given the high stakes of performance evaluations, statistical benchmarks should be accompanied by precision estimates. © 2011 Royal Statistical Society.