Sea-ice extent provides a limited metric of model performance

Abstract

We examine the common practice of using sea-ice extent as the primary metric to evaluate modeled sea-ice coverage. Based on this analysis, we recommend a possible best practice for model evaluation. We find that for Arctic summer sea ice, model biases in sea-ice extent can be qualitatively different compared to biases in the geophysically more meaningful sea-ice area. These differences come about by a different frequency distribution of high-concentration sea-ice: while in summer about half of the CMIP5 models and satellite retrievals based on the Bootstrap and the ASI algorithm show a compact ice cover with large areas of high concentration sea ice, the other half of the CMIP5 models and satellite retrievals based on the NASA Team algorithm show a loose ice cover. The different behaviour of the CMIP5 models can be explained by their different distribution of excess heat between lateral melt and sea-ice thinning. Differences in grid geometry and round-off errors during interpolation only have a minor impact on the different biases in sea-ice extent and sea-ice area. Because of regional cancellation of biases in the integrative measures sea-ice extent and sea-ice area, these measures show little correlation with the more meaningful mean absolute bias in sea-ice concentration. Comparing the uncertainty arising directly from the satellite retrievals with those that arise from internal variability, we find that the latter by far dominates the uncertainty estimate for trends in sea-ice extent and area: much of the differences between modeled and observed trends can simply be explained by internal variability. Only for the absolute value of sea-ice area, differences between observations and models are so large that they cannot be explained by either observational uncertainty nor internal variability.