View Synthesis Using Probabilistic Reliability Reasoning for FTV

Abstract

View synthesis using depth maps is a crucial application for Free-viewpoint TV (FTV). The depth estimation based on stereo matching is error-prone, leading to noticeable artifacts in the synthesized new views. To provide high-quality virtual views for FTV, we innovatively introduce a probabilistic framework that constrains the reliability of each synthesized pixel by Maximizing Likelihood (ML). Our spatial adaptive reliability is provided by incorporating Gamma hyper-prior and the synthesis error approximation using reference crosscheck¹⁾. Furthermore, we formulate view synthesis in the framework of Maximum a Posterior (MAP). For the outputs, two versions of the synthesized view are generated: the solution with ML criterion and the solution with MAP criterion, solved by straightforward interpolation and graph cuts, respectively. We experimentally demonstrate the effectiveness of both solutions with MPEG standard test sequences. The results show that the proposed method outperforms state-of-the-art depth based view synthesis methods, both in terms of subjective artifact reduction and objective PSNR improvement.