The ACCENT-protocol: A framework for benchmarking and model evaluation

Abstract

We summarise results from a workshop on"Model Benchmarking and Quality Assurance" of the EU-Network of Excellence ACCENT,including results from other activities (e.g. COST Action 732) and publications.A formalised evaluation protocol is presented, i.e. a generic formalism describing the procedure of howto perform a model evaluation. This includes eight steps and examples from global modelapplications whichare given for illustration.The first and important step is concerning the purpose of the model application,i.e. the addressed underlying scientific orpolitical question. We give examples to demonstrate that there isno model evaluation per se, i.e. without a focused purpose. Model evaluation is testing, whether amodel is fit for its purpose. The following steps are deduced from the purpose andinclude model requirements, input data,key processes and quantities, benchmark data, quality indicators, sensitivities, as well as benchmarking and grading.We define "benchmarking" as the process of comparing the model output against either observational data or high fidelity modeldata, i.e. benchmark data.Special focus is given to the uncertainties, e.g. in observational data, which have the potential tolead to wrong conclusions in the model evaluation if not considered carefully. © Author(s) 2012.