Abstract
This paper proposes a method that improves the quality of omnidirectional free-viewpoint images by generative adversarial networks. Omnidirectional images are a popular way of obtaining threedimensional (3D) visual information, while free-viewpoint images are essential to Virtual Reality (VR) and Mixed Reality (MR) applications. Therefore, we generated free-viewpoint images with 3D information estimated by the captured omnidirectional images. The quality of the generated images is deteriorated by the 3D reconstruction error due to occlusion and miss-correspondences. In this work, we proposed a method that uses Generative Adversarial Networks (GAN) to solve this problem. We focused on the structural information of various perspectives and applied a “divide and conquer” approach by separating the images into perspectives before training and recombining them at a later stage. At the same time, we conducted a comprehensive, multi-faceted evaluation of the proposed method to verify its effectiveness in improving image quality. Based on the actual information distribution in the equirectangular images, we analyze the adaptability of different image quality evaluation methods. After careful assessment, we consider that the proposed method can generate highly accurate, omnidirectional free-viewpoint images.
