3D Hand controller for playing game via body worn stereo camera

Abstract

Free hand motion in 3D space is natural and e(r)ective to handle virtual objects in cyberspace for game playing, etc. In this paper, we propose a 3D hand controller for game playing via stereo cameras that are attached on human body. This controller captures disparity image and color image of the hand of user by using stereo cameras on the user's body. From the images, 3D position and orientation of the hand relative to the body can be estimated. The orientation of the body relative to the flat plane in background scene can also be estimated. In accordance with the estimated values, the user can change motion and rotation of virtual viewpoint virtually, and manipulate virtual objects in cyberspace. For evaluating the e(r)ectiveness of the proposed hand controller, we apply it to control some 3D games.