Real-Time Free-Viewpoint Image Synthesis Using Multi-View Images and on-the-Fly Estimation of View-Dependent Depth Map

Abstract

The aim of our study was to synthesize free-viewpoint images without offline processing using input from multiple cameras that are configured in a 2D array. A set of depth layers is used to represent a target scene. We need to know to which layer each pixel on the synthesized image belongs. Acquiring this information is the same as estimating a pixel-by-pixel depth map that depends on the viewpoint from where the image is to be synthesized. In this paper, we describe an efficient method for estimating this depth map on-the-fly by verifying the color consistency of the input images. We then describe an efficient implementation method that uses acceleration of the graphics hardware and show that we can generate high-quality images at interactive frame rates. A quantitative evaluation to support the effectiveness of our method is presented.