Mowing detection intercomparison exercise (MODCiX) – Evaluation of grassland mowing detection algorithms across Europe

Abstract

Grasslands deliver a variety of ecosystem services, such as the provision of biomass, carbon sequestration or water retention and with that play a key role for climate change mitigation and preservation of biological diversity. The intensity of grassland management directly impacts these ecosystem services and functions. However, spatial information on grassland use intensity is scarce. Remote sensing time series from optical and/or SAR sensors help to overcome this data scarcity, as they enable to derive dates and frequency of mowing events as a proxy of grassland use intensity. A growing number of published algorithms relate abrupt changes in remote sensing time series to grassland management activities either by defining threshold-based rules or by making use of machine/deep learning techniques. So far, the different algorithms have not been compared and, due to a lack of suitable reference data, have usually not been tested for spatial and temporal transferability. We present the results of a comparison exercise based on an unprecedented set of independent reference data, containing information on more than 5000 grassland mowing dates that were compiled from eight European countries over a five-year period. We analyzed the performance of ten mowing detection algorithms across different geographic regions, years, mowing intensity, levels of reference data quality, and method and satellite sensor domains. The overall results show that when using all available reference data, F1 scores ranged from 0.55 to 0.74, and from 0.56 to 0.71 when only the highest quality reference data were considered. This decision, however, reduced the number of reference events to around 1500 with a regional bias to Austria, Germany, and Switzerland. We found that algorithm performance varies across space and time and that overall, the highest accuracies were achieved by machine learning based algorithms, although not substantially outperforming rule-based algorithms. The results did not confirm a consistently positive influence of the combined use of optical and SAR data in the prediction of mowing events, but we observed variations in algorithm performance, towards the lower and higher ends of grassland use intensity. Despite testing a variety of algorithms from different method and sensor domains, we observed a general upper limit of model performance and could not identify one single algorithm that performed best in all cases. The results of this comparison exercise can guide practitioners to choose approaches and input data that are most suitable for their specific use-case. The comparison exercise also highlights the importance of consistent, representative, and reliable reference data. We therefore recommend maintaining and extending this baseline dataset for the evaluation of upcoming algorithms, Earth Observation missions, and derived products for comprehensive monitoring of grassland use intensity.