Depth-Image-Based-Rendering by Segmentationbased on Color Image and Depth Map

Abstract

Depth-image-based rendering (DIBR) is a technique for synthesizing novel views of a scene from an image and its depth map. Many DIBR methods in the literature blurred the depth map before 3D warping to make the holes become smaller. However, the PSNR values when using methods with depth map blurring are lower than those when using methods without blurring. On the other hand, conventional methods without blurring failed when the scene was a complex 3D structure. In this paper, a DIBR method without depth map blurring is proposed for synthesizing novel views by 3D warping followed by depth map and hole interpolation. To interpolate holes after 3D warping naturally, it is important to recognize the objects surrounding holes especially when the scene structure is complex. To recognize objects in the image, the input image is segmented into small regions based on color and depth. Each hole is extrapolated by extending the surrounding small regions of the hole while considering the visibility and continuity. The experimental results demonstrate the efficiency of our method.