Evaluating observed versus predicted forest biomass: R-squared, index of agreement or maximal information coefficient?

Abstract

The accurate prediction of forest above-ground biomass is nowadays key to implementing climate change mitigation policies, such as reducing emissions from deforestation and forest degradation. In this context, the coefficient of determination (R2) is widely used as a means of evaluating the proportion of variance in the dependent variable explained by a model. However, the validity of R2 for comparing observed versus predicted values has been challenged in the presence of bias, for instance in remote sensing predictions of forest biomass. We tested suitable alternatives, e.g. the index of agreement (d) and the maximal information coefficient (MIC). Our results show that d renders systematically higher values than R2, and may easily lead to regarding as reliable models which included an unrealistic amount of predictors. Results seemed better for MIC, although MIC favoured local clustering of predictions, whether or not they corresponded to the observations. Moreover, R2 was more sensitive to the use of cross-validation than d or MIC, and more robust against overfitted models. Therefore, we discourage the use of statistical measures alternative to R2 for evaluating model predictions versus observed values, at least in the context of assessing the reliability of modelled biomass predictions using remote sensing. For those who consider d to be conceptually superior to R2, we suggest using its square d2, in order to be more analogous to R2 and hence facilitate comparison across studies.