Deep learning-based detection of roadside vegetation: how vegetation indices boost performance

Abstract

This study investigates the enhancement of deep learning-based roadside vegetation detection using vegetation indices. Vegetation detection is crucial for maintaining clear sightlines for autonomous driving and as an indicator of potential drainage issues and road deterioration. A Faster R-CNN architecture was employed to analyze images from vehicle-mounted cameras, with three separate input types evaluated: standard RGB images, Excess Green Index (ExG) images, and Color Index of Vegetation Extraction (CIVE) images. In addition, an integrated approach was developed that combined these input types. The results demonstrate that the integrated approach consistently outperformed individual input-based detections, achieving the highest Average Precision (AP) in both validation and test datasets. CIVE-based detection showed the highest overall performance among single input types, particularly in the test dataset. Vegetation indices generally improved detection accuracy compared to the standard RGB input, especially for challenging scenarios. However, all input types struggled with small object detection, indicating an area for future improvement. The study also revealed varying levels of detection consistency, with RGB-based detection showing the highest consistency across data sets. These findings contribute to the advancement of roadside vegetation detection techniques and suggest potential applications in comprehensive road condition assessment, automated maintenance planning, and early detection of drainage problems, complementing existing crack and pothole detection methods.