Semantic Segmentation of Camera Images and Projection to Lidar Point Clouds

Abstract

In this paper, we propose a method to project semantic segmentation results from monocular camera images onto 3D lidar point clouds. Our method uses mmsegmentation trained models for semantic segmentation and image geometry for projection. In our experiments, we used NVIDIA Isaac Sim, and evaluated the semantic segmentation and the projection of the label information in multiple environments. We compared several models of semantic segmentation and confirmed that the label information was correctly projected onto the point clouds.