Using measurements for evaluation of black carbon modeling
The ever increasing use of air quality and climate model assessments to underpin economic, public health, and environmental policy decisions makes effective model eval- uation critical. This paper discusses the properties of black carbon and light attenuation and absorption observations that are the key to a reliable evaluation of black carbon model and compares parametric and nonparametric statistical tools for the quantification of the agreement between models and ob- servations. Black carbon concentrations are simulated with TM5/M7 global model from July 2002 to June 2003 at four remote sites (Alert, Jungfraujoch, Mace Head, and Trinidad Head) and two regional background sites (Bondville and Is- pra). Equivalent black carbon (EBC) concentrations are cal- culated using light attenuation measurements from January 2000 to December 2005. Seasonal trends in the measure- ments are determined by fitting sinusoidal functions and the representativeness of the period simulated by the model is verified based on the scatter of the experimental values rel- ative to the fit curves. When the resolution of the model grid is larger than 1◦×1◦, it is recommended to verify that the measurement site is representative of the grid cell. For this purpose, equivalent black carbonmeasurements at Alert, Bondville and Trinidad Head are compared to light absorp- tion and elemental carbon measurements performed at dif- ferent sites inside the same model grid cells. Comparison of these equivalent black carbon and elemental carbon mea- surements indicates that uncertainties in black carbon optical properties can compromise the comparison between model and observations. During model evaluation it is important to examine the extent to which a model is able to simu- late the variability in the observations over different integra- tion periods as this will help to identify the most appropri- ate timescales. The agreement between model and observation is accurately described by the overlap of probability distribution (PD) curves. Simple monthly median compar- isons, the Student’s t-test, and the Mann-Whitney test are discussed as alternative statistical tools to evaluate the model performance. The agreement measured by the Student’s t- test, when applied to the logarithm of EBC concentrations, overestimates the higher PD agreements and underestimates the lower PD agreements; the Mann-Whitney test can be employed to evaluate model performance on a relative scale when the shape of model and experimental distributions are similar.