Optimal Battery Sizing for Real-Time Renewable Energy Bidding Based on Reinforcement Learning

Abstract

With the increasing penetration of Variable Renewable Energy (VRE) sources such as photovoltaic and wind power, effective market integration of VRE is essential to minimize market distortions and support new business models. Battery energy storage systems enhance operational flexibility by mitigating imbalance penalties and enabling energy arbitrage. However, accurate battery sizing is challenging due to its strong coupling with operational strategies, making it a multi-scale optimization problem. Additionally, uncertainty in renewable generation and market prices increases the computational complexity, as is typically the case for stochastic programming approaches. This paper proposes a computationally efficient method for optimal battery sizing for renewable producers participating in real-time markets. We leverage Reinforcement Learning (RL) as a stochastic sequential decision-making framework and extend it to co-optimize the battery capacity with the bidding strategy. Unlike existing bi-level optimization approaches, our method integrates battery sizing directly into the bidding policy training process. This one-shot optimization framework improves computational efficiency by eliminating the need for repeated RL trainings across candidate battery capacities. Numerical simulations demonstrate that the proposed method effectively optimizes battery investment and operational performance over a target year.