Traffic light recognition using deep learning with a large-scale driving dataset for self-driving cars

Abstract

Traffic light recognition is critical to the realization and social implementation of self-driving cars. Traffic lights in Japan generally consist of three colors (red, yellow and green) and are often oriented horizontally, but the shape, size, and arrangement of colors vary depending on the country and city. With the advancement of image processing technology, traffic light recognition using deep learning-based object detection models has been researched. However, most large-scale datasets were collected overseas, and only a few were managed and constructed in Japan. Furthermore, it is still challenging to improve the accuracy of the object detection models and understand the context of recognized traffic lights and travel lanes. In this study, we collected more than 900 hours of driving data collected independently on Japanese public roads and constructed a dataset of 15,000 images of multiple scenes annotated with traffic lights. Using this dataset, we trained a deep learning-based 2D object detection model to recognize the traffic lights corresponding to the own driving lane. We verify that the proposed model is specialized to Japanese traffic data and is helpful for traffic light recognition in self-driving cars.