Abstract
In this paper, we introduce an intelligent pace-car in the traffic flow for the purpose of controlling vehicles that follows a pace-car, and show that it is possible to reduce the phantom traffic jam. Under the situation of phantom traffic jam, the instabilities are observed to grow into traveling waves, which are local peaks of high traffic density, although the average traffic density is still moderate, where the highway is not fully congested. The pace-car manages its velocity to control the following vehicles which are forced to brake when they run into such waves.
The management strategy of the velocity is acquired by reinforcement learning. We employ the extended Nagel-Schreckenberg model which make the traffic flow maximum. By introducing the learned pace-car, we successfully achieves a phase transition that shifts traffic flow from congestion phase to metastable phase.
First, we explain our proposed extended Nagel-Schreckenberg model, and secondly, it is defined that the situation of congestion as the state space to make pace-car learn by reinforcement learning approach. Third, though pace-car finally realizes the highest traffic flow, we evaluate the traffic loss during the period of pace-car's control. As the result. Finally, through the loss evaluation, we show the effectiveness of our approach to acquire the control strategy of pace-car.
