動画像からの頭部と身ぶりの運動追跡

抄録

In the present virtual reality system, the user must wear certain sensors (e.g. data gloves, head mounted display) in order to measure the user's head position and orientation and track other body parts such as the user's hand and arm. A more flexible interface paradigm measures the head position and orientation or gesture directly from images captured by video cameras; thus freeing the user from wearing the cumbersome sensors. However, the image processing technique is still experimental because the previous structure-from-motion techniques suffer from lack of robustness against noise, instability caused by sharp acceleration, and uncertainty due to line-of-sight occlusion. To addresses these drawbacks, we describe a Kalman filtering-based technique to recover more robustly 3-D structure and kinematics of the head and arm in view from optical flow. Because of its explicit modeling of measurement noise and modeling uncertainty, the extended Kalman filter (EKF) has been shown to improve the robustness of structure-from-motion techniques. The robustness of the recursive estimation technique is further enhanced by (1) confining feature-tracking within the neighborhood of skeleton sketch of the head and arm, (2) formulating measurement noise of the optical flow as a function of the optical flow's confidence measures, and (3) for the occluded feature points, fusing into EFF the optical flow predicted from the estimated 3-D kinematics, and (4) compensating the feature tracking errors with a feedback loop that tracks feature points from the projections of structural estimates.