Transactions of the Society of Instrument and Control Engineers
Online ISSN : 1883-8189
Print ISSN : 0453-4654
ISSN-L : 0453-4654
Dynamic Obstacles Detection and 3D Reconstruction by a Monocular Camera and a 3D-map
Author information

2021 Volume 57 Issue 1 Pages 37-46


We developed a new method for obstacles detection and 3D reconstruction using a 3D map. Obstacles detection and 3D reconstruction are key functions of autonomous driving. It is easy to detect and reconstruct static obstacles three-dimensionally because they exist in the 3D map. However, the detection and the 3D reconstruction of dynamic obstacles that are not in the 3D map is difficult for a typical in-vehicle camera that cannot measure the distance. We aim to detect dynamic obstacles three-dimensionally, using an in-vehicle camera. And we deal with the new problem of accurate 3D reconstruction by using a monocular camera and a 3D map. To solve this problem, we focused on semantic segmentation for detection and depth completion to complement the depth map. We propose a multi-task neural network (NN) that shares the encoder of semantic segmentation NN and depth completion NN, whose inputs are an image and the 3D map. The proposed multi-task NN detects dynamic obstacles 1.4 times more accurately than the single-task state-of-the-art method.

Information related to the author
© 2021 The Society of Instrument and Control Engineers
Previous article Next article