Vehicle and its Shadow Detection in MMS Images Using Deep Learning to Generate High-resolution Road Orthophotos

Abstract

In recent years, extensive researches have been conducted to automatically generate high-resolution road orthophotos using images and laser point cloud data collected by a Mobile Mapping System (MMS). However, it is necessary to detect and mask out the areas of non-road objects in MMS images such as vehicles, bicycles, pedestrians and their shadows, in order to eliminate erroneous textures from the road orthophotos. Hence, we proposed a novel vehicle and its shadow detection method based on Faster R-CNN for improving the detection accuracy, especially the accuracy of detected regions. The experimental results showed that the recall of the proposed method was 93.9% (Intersection-over-Union＞0.7), which was 7.0% higher than 86.9% obtained by Faster R-CNN. Moreover the proposed method could identify the regions of vehicles and their shadows accurately and robustly in MMS images, even though the images contained various types of vehicles, different shadow directions, and partial occlusions. Furthermore, it was confirmed that the quality of road orthophoto generated using vehicle and its shadow masks by the proposed method was significantly improved as compared to those generated using no masks, vehicle masks and even the vehicle and its shadow masks by Faster R-CNN.