Traffic light control using hierarchical reinforcement learning and options framework

Abstract

The number of vehicles worldwide has grown rapidly over the past decade, impacting how urban traffic is managed. Traffic light control is a well-known problem and, although an increasing number of technologies are used to solve it, it still poses challenges and opportunities, especially when considering the inefficiency of the popular fixed-time traffic controllers. This study aims to apply Hierarchical Reinforcement Learning (HRL) and Options Framework to control a signalized vehicular intersection and compare its performance with that of a fixed-time traffic controller, configured using the Webster Method. HRL combines the ability to learn and make decisions while taking observations from the environment in real-time. These capabilities bring a significant adaptive power to a highly dynamic problem. The test scenarios were built using the SUMO simulation tool. According to our results, HRL presents better performance than those of its own isolated sub-policies and the fixed-time model, indicating a simple and efficient alternative.