Evaluating Accuracy and Temporal Consistency of Machine Learning Models for Land Use/Land Cover Mapping in the Cimanuk Watershed

Abstract

One of the critical watersheds in Indonesia is the Cimanuk watershed, which is recognized as a national priority watershed. Remote sensing plays a vital role in monitoring land use/land cover (LULC) in the Cimanuk watershed. Overall accuracy (OA) and Kappa coefficient (KC) are commonly used as primary measures of classification accuracy. In addition to these parameters, it is essential to examine the consistency of LULC classification over time. This study aimed to compare the accuracy and temporal consistency of three Machine Learning Models: random forest (RF), support vector machine (SVM), and classification and regression tree (CART) for LULC classification in the Cimanuk watershed using Sentinel-2 Multispectral Instrument (MSI) images for 2020 and 2025, processed in Google Earth Engine (GEE). The results indicate that RF has the highest accuracy, with 87.9% (OA) and 83.7% (KC) in 2025, and 83.6% (OA) and 77.9% (KC) in 2020. The lower classification accuracy in 2020 compared to 2025 is likely due to the 2025 image having clearer visual characteristics and better spectral separability among LULC classes. When examining inconsistent transitions between 2020 and 2025, CART accounted for 11.97% of these transitions, higher than RF and SVM. After applying temporal consistency correction to the 2020 LULC classification result, RF remains the best-performing classifier, achieving 90% (OA) and 86.5% (KC), followed by CART and SVM. These findings provide valuable insights into incorporating accuracy and temporal consistency assessments into time-series LULC analysis, serving as a reference for future LULC studies in watershed management and other geographic contexts.