Extended contingency table: Performance metrics for satellite observations and climate model simulations

Abstract

Validation of gridded satellite observations and climate model simulations are fundamental to future improvements in retrieval algorithms and model developments. Among the metrics, the contingency table, which includes a number of categorical indices, is extensively used in evaluation studies. While the categorical indices offer invaluable information, they do not provide any insight into the volume of the variable detected correctly/incorrectly. In this study, the contingency table categorical metrics are extended to volumetric indices for evaluation of gridded data. The suggested indices include (a) Volumetric Hit Index (VHI): volume of correctly detected simulations relative to the volume of the correctly detected simulations and missed observations; (b) Volumetric False Alarm Ratio (VFAR): volume of false simulations relative to the sum of simulations; (c) Volumetric Miss Index (VMI): volume of missed observations relative to the sum of missed observations and correctly detected simulations; and (d) the Volumetric Critical Success Index (VCSI). The latter provides an overall measure of volumetric performance including volumetric hits, false alarms, and misses. First, using two synthetic time series, the volumetric indices are evaluated against the contingency table categorical metrics. Then, the volumetric indices are used to evaluate a gridded data set at the continental scale. The results show that the volumetric indices provide additional information beyond the commonly used categorical metrics that can be useful in evaluating gridded data sets. Key Points Volumetric metric provide valuable information beyond categorical metrics Categorical metrics of the contingency table are extended to volumetric measures Presented metrics can be used for validation of all types of gridded data ©2013. American Geophysical Union. All Rights Reserved.