A score for probability forecasts of binary events based on the user’s cost–loss relations

Abstract

A score for verifying probabilistic forecasts is presented. It is called the continuous specific score (CSS) and is intended for binary events only. The score is based on the user’s cost–loss relations and their relative importance. The relative importance is determined by a continuous function of the user’s density of loss for various cost–loss relations. One may also consider CSS as the result of the expected mean value based on a probability function of the various loss values for all possible cost–loss relations for one single user. The CSS is a negatively oriented score and has the following properties: perfect forecasts yield the score zero, and 100% probability for adverse weather (AW) when AW does not occur leads to the score one. The result of a 0% forecast of AW when AW occurs is the inverted value of the ratio of the average cost to the average loss minus one. Different possible usages of the score are discussed. An effective cost–loss ratio (ECLR) is defined. It measures how important low cost–loss ratios are compared to higher ones.