[Paper] Free-viewpoint Video Synthesis for Sport Scenes Captured with a Single Moving Camera

抄録

In this paper, we propose an algorithm to realize an immersive free-viewpoint experience, whereby audiences can see real sport scenes from any viewpoint, from a single moving camera. Homography matrix estimation and object extraction are two of the most important processes for synthesizing free-viewpoint video, since a world coordinate position of each object needs to be calculated in every frame based on object textures and homography matrix between captured frame and 2-dimensional world coordinate model of sports coat. Most conventional methods only use static cameras whose homography matrices and background models do not change. However, broadcasters usually utilize moving pan-tilt-zoom cameras in sporting events, and therefore, those methods cannot be directly applied to the events. In order to extend the range of application of free-viewpoint video technology to be used in sport live broadcasting, our proposed method estimates homography matrices semi-automatically by identifying reliable corresponding feature points between video frames, and also extracts the precise textures of objects using estimated homograph matrices. Experimental results revealed that the proposed method successfully estimated the precise homography matrices compared to the conventional method. Moreover, it was also confirmed that the proposed scheme contributes to further improvement in the experience of free-viewpoint video since the textures of multiple objects were successfully extracted.