Development of an Information Projection Interface using a Projector-Camera System

Abstract

This paper proposes an interface system that enables a user to input and obtain information position-freely within a certain area of the environment with a projector-camera system. Hand-waving detected in the image is used as a cue to start the system, to distinguish gestures based on the user's intension of operation from other daily-life motions, and to specify the position of the user's hand. Hand-waving detection is implemented based on combination of background subtraction method and skin color extraction. In addition, we focus on changes of brightness value in each pixel. The user's hand is tracked using a particle filter algorithm. The user designates the projection area freely on a plane in the environment with his/her hand. The projector mounted on a pan-tilt unit projects information according to the user's designation. Then, the user's fingertip is detected using simple template matching and he/she interacts with the projected information to input commands without any devices. As one of applications, we have developed an interface system to operate electrical appliances (TV) intuitively in a daily life scene.