3D Reconstruction Based on Style Transfer Data Augmentation

Abstract

Recently, various technologies using PointCloud and Deep Neural Network (DNN) have been actively researched. However, there is a disadvantage that the collecting PointCloud data from real object with special sensors such as depth sensor is time consuming task. To deal with this problem, we focus on 3D reconstruction from a single image. Conventional methods construct PointCloud from a single image which includes mask information. Therefore, it is difficult to construct a PointCloud from an image without mask information. To remove the requirement of the additional information such as mask for input image, we propose data augmentation based on style transfer for 3D reconstruction. It is known that DNN using style transformed image can learn a shape feature. By using the transformed images, the DNN can learn object shapes with various backgrounds and textures and can obtain shape features even from the images with cluttered background. From the experimental results, we confirmed that our proposed method could construct 3D object shape with PointCloud while keeping shape information without additional information.