A new set of indicators for model evaluation complementing FAIRMODE's modelling quality objective (MQO)

Abstract

In this study, we assess the relevance and utility of several performance indicators (model quality (bias) and model performance (temporal and spatial) indicators), developed within the FAIRMODE framework by evaluating eight Copernicus Atmospheric Monitoring Service (CAMS) models and their ensemble in calculating concentrations of key air pollutants, specifically NO2, PM2.5, PM10 and O3. The models’ outputs were compared with observations that were not assimilated into the models. For NO2, the results highlight difficulties in accurately modelling concentrations at traffic stations, with improved performance when these stations are excluded. While all models meet the established criteria for PM2.5, indicators such as bias and winter–summer gradients reveal underlying issues in air quality modelling, questioning the stringency of the current criteria for PM2.5. For PM10, the combination of model quality indicators, bias, and spatial-temporal gradient indicators prove most effective in identifying model weaknesses, suggesting possible areas of improvement. O3 evaluation shows that temporal correlation and seasonal gradients are useful in assessing model performance. Overall, the indicators provide valuable insights into model limitations, yet there is a need to reconsider the strictness of some indicators for certain pollutants.