A Note on the Ranked Probability Score

Abstract

In this paper, we compare the ranked probability score (RPS) and the probability score (PS) and examine the nature of the sensitivity of the RPS to distance. First, we briefly describe the nature of and the relationship between the frameworks within which the RPS and the PS were formulated. Second, we consider certain properties of the RPS and the PS including their range, their values for categorical and uniform forecasts, and their proper nature. Third, we describe the RPS and the PS in a manner that reveals the structure of and the relationship between these scoring rules. Fourth, we considered the RPS with reference to two definitions of distance and examine the nature of the sensitivity of the RPS to distance. The comparison of the RPS and the PS suggests that the RPS rather than the PS should be used to evaluate probability forecats, at least in those situations in which the variable of concern is ordered.Supported in part by the National Science Foundation (Atmospheric Sciences Section) under Grant GA-1707Contribution Number 171 from the Department of Meteorology and Oceanography, The University of Michigan