Abstract
In this paper, we propose a robust object tracking scheme for multi-view cameras and consecutive frames for rendering an immersive free-viewpoint video in a large outdoor space such as a soccer stadium. For a free-viewpoint video that provides users with an immersive experience, each object has to be identified consistently among all cameras for every frame to share the textures of the same objects and replace the textures when an occlusion occurs. To satisfy this requirement, the proposed method extracts objects' silhouette regions and tracks each identified object by associating a closed silhouette region with a tracking ID for every camera. During the frame by frame process, our method confirms whether occlusion occurs for each tracking region and modifies the texture region by projecting the world coordinate of the object in 3D-space, which can be estimated from a camera image without occlusion if one is available. The experimental results revealed that the proposed method achieved more robust texture extraction of multiple objects especially for occluded regions compared to the conventional methods. Furthermore, it was confirmed that the proposed scheme can improve the subjective image quality for free-viewpoint video as a result of precise reconstruction of occluded regions.
