Decentralized Traffic Signal Control Following Changes in Traffic Flow via Hierarchical Reinforcement Learning

Abstract

Efficient traffic signal control (TSC) must consider the optimization over a large area and the computational cost associated with their control range. Autonomous decentralized control can solve this problem. Deep reinforcement learning (DRL) based control has been studied extensively recently as a new alternative to rule-based control. However, DRL learns optimal policies (control laws) for experienced environments. Therefore, it only guarantees performance under the unexperienced traffic flow. Furthermore, multiple optimal policies exist because of a trade-off between orthogonal traffic flows at intersections. We solve these two problems via a hierarchical TSC method consisting of two types of agents: (1) collect an exhaustive set of policies for each traffic flow ratio, and (2) select a suitable policy considering the traffic conditions. Computer experiments show that the proposed method has the flexibility to switch policies in response to changes in traffic flow.