Partial Orderings of Default Predictions

Abstract

We compare and generalize various partial orderings of probability forecasters according to the quality of their predictions. It appears that the calibration requirement is quite at odds with the possibility of some such ordering. However, if the requirements of calibration and identical sets of debtors are relaxed, comparability obtains more easily. Taking default predictions in the credit rating industry as an example, we show for a database of 5333 (Moody’s) and 6505 10-year default predictions (S&P), that Moody’s and S&P cannot be ordered neither according to their grade distributions given default nor non-default or to their Gini- curves, but Moody’s dominate S&P with respect to the ROC-criterion.