Achieving Long-Term Autonomy: A Self-Correcting Deep Reinforcement Learning Agent for Edge IoT Using Digital Twin-Based Drift Compensation

Abstract

Ensuring long-term autonomy in Edge AI systems remains one of the most persistent challenges in environmental monitoring and biorisk management. Over time, the degradation of low-cost sensors—particularly sensor drift—leads to cumulative measurement errors, distorted state perception, and catastrophic decision failures in Deep Reinforcement Learning (DRL) agents. This paper proposes a novel Self-Correcting Deep Reinforcement Learning (SCDRL) framework that enables robust, long-term autonomy through in-loop drift compensation. The proposed Self-Correcting Agent (SCA) integrates a dualinput architecture combining (i) the local, drifted sensor reading and (ii) a stable reference prediction from a macro-scale Digital Twin (DT). By learning to correlate both signals, the agent implicitly estimates and neutralizes sensor bias in real time, achieving self-calibration without human intervention. To validate this approach, a nine-year simulation of autonomous water management was conducted using real-world hourly climate data from Arequipa, Peru. Results show that a conventional “blind” DRL agent suffers complete performance collapse as drift accumulates, whereas the proposed SCA maintains stable operation indefinitely. Quantitatively, the SCA achieved a 722% higher cumulative reward (415,662 vs. 57,556) and a 53% reduction in plant stress (RMSE 0.2238 vs. 0.4762). These findings establish a validated blueprint for fault-tolerant Edge AI, demonstrating that the fusion of local sensing with digital twin predictions enables self-calibrating agents capable of sustained, reliable autonomy in real-world, resource-constrained environments.