The boy who cried wolf revisited: The impact of false alarm intolerance on cost-loss scenarios

Abstract

Meteorologists often interpret the value of a probabilistic weather forecast using the binary cost-loss scenario. The socioeconomic benefit of such a forecast will depend on the compliance rate of users and, hence, the number of warnings that are not followed by the corresponding high-impact weather. A modified version of the canonical binary cost-loss problem in which the compliance rate of users is a function of the warning probability threshold, and hence of the "false alarm rate", is presented. In this version of the problem, the value of the forecast can be enhanced by choosing a probability warning threshold that is higher than the cost-loss ratio. It is found that the advantage of modifying the probability warning threshold is greatest when the frequency of highly confident forecasts of an event is relatively high and when users are moderately intolerant of false alarms. Using this simple example it is illustrated that forecasters who issue nonprobabilistic, or "unequivocal," forecasts are making implicit assumptions about the false alarm intolerance of users, as well as assumptions about their costloss ratios. Adopting a probabilistic approach to forecasting avoids these assumptions and separates the activity of forecasting from the activity of decision making. © 2004 American Meteorological Society.