Three-way ROC analysis using SAS software

Abstract

The most commonly used measure of model accuracy in medicine with three categories of target variable is the volume under ROC surface (VUS), which is the extension of the area under curve (AUC) for binary models (Le and Lili, 2013). This paper deals primarily with usage of the multinomial logistic regression and the three-way ROC analysis in the financial sector, especially in the credit risk management. Moreover, SAS system is very often used software in the financial sector. Therefore this paper is focused on ways of doing three-way ROC analysis in this statistical software, in particular on estimating the VUS. We propose an estimate of the VUS based on the confusion matrix, which is compared to estimates based on Mann-Whitney statistic and on empirical distribution functions. We developed three SAS macros based on these approaches for computing the VUS. Further- more, we developed some logistic models for three-value target variable based on the Loss Given Default (LGD). This was done on real financial data. Results obtained by the SAS macros on these models are presented a discussed in the paper.