Reproduction of Highlights for Hiding an Augmented Reality Marker on a Plane

Abstract

In augmented reality (AR) applications, a marker is often used for its robustness for camera pose estimation and its easiness for placing virtual objects on user's desired positions. However, AR markers in video may prevent from seamlessly fusing real environments and virtual objects. Therefore, plausibly hiding AR markers is desired in AR applications. As technology that can be applied to marker hiding, diminished reality (DR) has been investigated to visually remove real objects from video in real-time by generating plausible background images on them. However, conventional DR methods cannot correctly reproduce background images with highlights on a specular surface. This paper proposes a method that plausibly removes AR markers on a specular surface by estimating an illumination environment from images and reproducing highlights on a background image according to camera poses. Experiments show the effectiveness of the proposed method by placing an AR marker on various environments and comparing results obtained by the proposed and conventional methods.