A Simulation Study on Coordinated Urban Traffic Signal Control Using Multi-Agent Reinforcement Learning

Abstract

Urban traffic congestion poses a significant challenge to modern city management. To address the limitations of traditional fixed-time signal control strategies, which lack the flexibility to adapt to dynamic traffic flows, this paper proposes a coordinated control framework based on Multi-Agent Proximal Policy Optimization. In this framework, each intersection traffic signal is regarded as an intelligent agent that learns optimal signal timing strategies by interacting with the environment and collecting traffic state data. To tackle the coordination challenges among agents, we designed an attention-based communication mechanism, enabling agents to dynamically share critical intersection information. Furthermore, we introduced a hierarchical curriculum learning strategy that guides agents to learn complex control policies robustly and efficiently by automatically generating training scenarios of increasing difficulty. Extensive experiments conducted on a typical urban road network built within the open-source simulation platform SUMO demonstrate that our method significantly outperforms traditional fixed-time control, actuated control, and independent reinforcement learning methods in key metrics such as average vehicle delay, queue length, and traffic efficiency. This research provides a novel intelligent approach and a reproducible simulation benchmark for solving urban traffic optimization problems.