Photorealistic Augmented Reality Image Generation with Generative Adversarial Network Focusing on Structural Edge

Abstract

In order to represent virtual objects photorealistically in augmented reality (AR), the problem of optical consistency is important. There are several methods to achieve optical consistency using known real objects and special cameras, but they are difficult to use in AR applications. In this research, we propose an end-to-end method to convert an optically inconsistent AR image into an optically consistent AR image using a generative adversarial network (GAN). In addition, we propose a GAN that focuses on the structural edges of virtual objects in order to be able to handle different virtual object shapes. We confirmed that the GAN can generate photorealistic AR images consistent with the real world and that it is possible to generate images with versatility for virtual object shapes.