Optimization of Traffic Signal Cooperative Control with Sparse Deep Reinforcement Learning Based on Knowledge Sharing

Abstract

Urban traffic management is highly complex, and inefficient control strategies often worsen congestion and increase energy consumption. This paper introduces a collaborative multi-agent reinforcement learning method tailored for sparse control scenarios, IKS-SAC (Improved Knowledge Sharing Soft Actor–Critic), which enhances coordination between traffic signals to optimize traffic flow. IKS-SAC incorporates a communication protocol for knowledge sharing among agents, enabling each agent to access and utilize traffic environment data collected by other agents, effectively addressing the challenge of data processing in asynchronous updates, thereby achieving a comprehensive understanding of the traffic environment within a sparse control framework. Validation of the synthetic data demonstrates that IKS-SAC exhibits superior adaptability and efficiency in managing traffic flow fluctuations and uncertainties, significantly outperforming existing reinforcement learning-based and traditional traffic control methods. The proposed method demonstrates significant advantages in reducing traffic congestion, lowering energy consumption, and mitigating environmental pollution.