Q-Learning–Based Coordinated Energy Management and Double Auction Trading for Battery Swapping Charging Stations With PV and VPP: A Multiscenario Techno-Economic Study

Abstract

Electric vehicles (EVs) are increasingly recognized for their cost-effectiveness and environmental benefits in reducing transportation-related emissions. However, growing EV adoption, combined with renewable energy variability, poses significant challenges for local electrical grids and charging infrastructure operations. This article proposes a market-integrated Q-learning framework for the coordinated operation of battery swapping and charging stations (BSCSs) interacting with virtual power plants (VPPs) under renewable integration and market uncertainty. Unlike conventional reinforcement-learning (RL)–based EV charging strategies that assume static tariffs and passive price-following behavior, the proposed approach explicitly incorporates dynamic energy trading, battery degradation, and multisource uncertainty into the decision-making process. The BSCS jointly optimizes charging, swapping, and market transactions while responding to stochastic EV demand, photovoltaic (PV) generation variability, and electricity price volatility under time-of-use (ToU) pricing. A comprehensive Monte Carlo analysis with correlated uncertainties demonstrates robust power flow regulation and stable profitability across various techno-economic scenarios. Long-term battery degradation and state-of-health evolution over multiyear horizons are incorporated, revealing marginal economic impact without altering optimal policy structure. Sensitivity and convergence analyses confirm algorithmic robustness across learning parameters, while scalability studies validate applicability from single-station to multistation deployments. The results demonstrate that market-aware RL enables economically viable peak demand handling, profit maximization, and sustainable operation of EV charging infrastructure. This work highlights how VPP integration can enhance energy management and trading, reduce operational costs, and promote resilient energy solutions under realistic and strategically relevant conditions. Moreover, the framework aligns with policy objectives related to grid stability, renewable integration, and sustainable mobility infrastructure (SDG 7 and SDG 11).