Super-Resolved Free-Viewpoint Image Synthesis Based on View-Dependent Depth Estimation

Abstract

We present a method for synthesizing high-quality free-viewpoint images from a set of multi-view images. First, an accurate depth map is estimated from a given target viewpoint using modified semi-global stereo matching. Then, a high-resolution image from that viewpoint is obtained through super-resolution (SR) reconstruction. The depth estimation results from the first step are used for the second step in two ways. First, the depth values are used to associate pixels between the input images and the latent high-resolution image. Second, the pixel-wise reliabilities of the depth information are used for regularization to adaptively control the strength of the SR reconstruction. Extensive experimental results using real images show the effectiveness of our method.