Deep Learning and 3D Capture / 3D Model Generation

Abstract

3D data is useful not only for design but also for digital twins, entertainment, and other applications. However, the barrier to creating 3D data using 3D modeling tools is high, and the resources required are significant. To address this, 3D capture and 3D model generation technologies are being explored. Among these, 3D capture, which mechanically captures real-world spatial information and converts it into 3D data, has seen significant improvements in quality through methods inspired by deep learning. Both deep learning-based image recognition and photogrammetry have succeeded not by ’going through several independently designed modules to reach the output,’ but by ’directly focusing on the desired output and optimizing the recognition model/3D model,’ which has been the key to their success. The technology of generating new 3D models through linguistic instructions is an extension of image generation but is still not fully mature, and performance improvements are expected in the future.