We developed a new method for obstacles detection and 3D reconstruction using a 3D map. Obstacles detection and 3D reconstruction are key functions of autonomous driving. It is easy to detect and reconstruct static obstacles three-dimensionally because they exist in the 3D map. However, the detection and the 3D reconstruction of dynamic obstacles that are not in the 3D map is difficult for a typical in-vehicle camera that cannot measure the distance. We aim to detect dynamic obstacles three-dimensionally, using an in-vehicle camera. And we deal with the new problem of accurate 3D reconstruction by using a monocular camera and a 3D map. To solve this problem, we focused on semantic segmentation for detection and depth completion to complement the depth map. We propose a multi-task neural network (NN) that shares the encoder of semantic segmentation NN and depth completion NN, whose inputs are an image and the 3D map. The proposed multi-task NN detects dynamic obstacles 1.4 times more accurately than the single-task state-of-the-art method.
