Effective Use of Planar Structure of Hidden Area and Surrounding Region of Interest in Diminished Reality(<Special Issue>Mixed Reality)

Abstract

In this paper, we propose a diminished reality (DR) system by which target objects in front of planar structure are effectively removed from the user's view. Our approach is to use a view-dependent image set captured before target objects are placed in an environment and to use common region (surrounding region of interest; SROI) between the view-dependent images and the user's view. First, in the proposed method, a target object in the user's view is overlaid by a selected view-dependent image using homography warping. For accurately projecting the view-dependent image, feature points are matched to ones only in SROI. During real-time process, lightings can change and can be different from that in the advanced capture of the view-dependent image set. Therefore, the projected view-dependent image is compensated using color information in SROI. In experiments, we used a 6DOF robot arm to capture ground truth image sequences, and compared them with our DR results. The quantitative results showed success and limits of the proposed method.