Mixed Reality System that Displays Foreground Objects Semi-transparently Using Depth Information

Abstract

In the field of the mixed reality technologies, the systems that visualize foreground and occluded background simultaneously by displaying occluders (foreground objects) semi-transparently have been proposed. In this study, we propose to improve such systems by using depth information captured from an RGB-D camera. In the proposed system, by using the depth information, the foreground objects are extracted from the background more accurately than the conventional systems. The proposed system achieves robust recognition of the foreground objects against textures and illumination conditions, and updates of the occluded background by only a single RGB-D camera. Furthermore, the proposed system also achieves variable transparency of objects according to the depth from the background. The foreground object nearer to the background is displayed more thickly. This method can show the positional relationship between the foreground objects and the background to users more clearly, and assists their works. We carry out some experimental subjective evaluations of the proposed system. Experimental results show the effectiveness of the system.