IALight: Importance-Aware Multi-Agent Reinforcement Learning for Arterial Traffic Cooperative Control

Abstract

Multi-intersection cooperative control for arterial or network scenarios is a crucial issue in urban traffic management. Multi-agent reinforcement learning (MARL) has been recognised as an efficient solution and shows outperformed results. However, most existing MARL-based methods treat intersection equally, ignoring different importance of each intersection, such as high traffic volume, connecting multiple main roads, serving as entry or exit point for highways or commercial areas, etc. Besides, learning efficiency and practicality remain challenges. To address these issues, this paper proposes a novel importance-aware MARL-based method named IALight for traffic optimisation control. First, a normalised traffic pressure is introduced to ensure our state and reward design can accurately reflect the status of intersection traffic flow. Second, a reward adjustment module is designed to modify the reward based on intersection importance. To enhance practicality and safety for real-world applications, we adopt a green duration optimisation strategy under a cyclic fixed phase sequence. Comprehensive experiments on both synthetic and real-world traffic scenarios demonstrate that the proposed IALight outperforms the traditional and deep reinforcement learning baselines by more than 20.41% and 17.88% in average vehicle travel time, respectively.