Estimation of Omni-Directional Pointing Gestures using Multiple Cameras

Abstract

We propose a multi-camera system that can detect omni-directional pointing gestures and estimate the direction of pointing. In general, a pointing target exists in the face direction when a human points to some-thing. Therefore, we regard the direction of pointing as the straight line that connects the eye position with the finger tip position. First, the multiple cameras detect the face region by use of skin color and estimate the face direction with the discrete face direction feature classes. Second, we estimate the precise direction that the subject is facing with the integrated information from multiple cameras and find which camera captures the frontal view of the face the best. This camera is labeled the center camera. Third, we select the center camera and its neighboring cameras and detect the spatial positions of the eye and finger tip in stereo mode. Finally, the target that the subject is pointing to is found on the straight line that connects the eye position with the finger tip position. Experiments show that our system can achieve a mean error of 0.97° and a variance of 1.48 for all directions of the pointing.