THE RANKED PROBABILITY SCORE AND THE PROBABILITY SCORE: A COMPARISON 1,, 2

Abstract

In this paper, we compare the ranked probability score (RPS) and the probability score (PS) and examine the nature of the sensitivity of the RPS to distance. First, we briefly describe the nature of and the relationship between the frameworks within which the RPS and the PS were formulated. Second, we considsr certain properties of the RPS and the PS including their range, their values for categorical and uniform forecasts, and their “proper” nature. Third, we describe the RPS and the PS in a manner that reveals the structure of and the relationship between these scoring rules. Fourth, we consider the RPS with reference to two definitions of distance and examine the nature of the sensitivity of the RPS to distance. The comparison of the RPS and the PS suggests that the RPS rather than the PS should be used to evaluate probability forecasta, at least in those situations in which the variable of concern is ordered.