Assessing the suitability of sets-based approaches: estimating the discriminative power of risk models for ordinal outcome treatments

Abstract

BACKGROUND: In order to evaluate the discrimination performance of an ordinal model for improved disease screening, a new test was proposed where information was obtained across all samples simultaneously. METHODS: The ordinal c-index builds upon the volume under the surface methodology without focusing on the accompanying receiver operating characteristic surfaces. However, it can be simplified to an average of pairwise c-indexes. In this paper, a set-based estimate (information was obtained across all samples simultaneously) was proposed by summing all correctly ordered groups. The asymptotic distribution of this proposed estimate was derived using U-statistics. RESULTS: A predictive model was applied using the blood urea nitrogen/creatinine ratio to discriminate stroke in evolution in acute ischemic stroke patients, which could potentially be life-saving in emergency departments. CONCLUSIONS: By conducting Monte Carlo simulations, it was concluded that the measure proposed herein is a better choice for clinical use because of the asymmetry of the predicted probabilities of groups.