Pointing Input Method robust to Small Background Changes for Information Projection Interface

Abstract

We propose a methodology for pointing-gesture detection using background subtraction that is robust to small changes in the background. We have developed an interface system that enables the user to input and obtain information in a position-free manner within a specific area of the environment using a projector-camera system. The system projects a command dialog box and observes the button that the user points at with their fingertip and thus receives commands. To detect the pointing gesture, the system extracts the hand region using background subtraction and detects the fingertip position based on the convex hull of the hand. We studied a background subtraction methodology that does not require a model of the background changes in advance. First, phase-only correlation is used to align positions between the input image and the background image. Then, similarities calculated from three features; local binary patterns, fractal dimension and color, are integrated into a unified similarity by Choquet integral. Noise is removed using a temporal weighing map representing the possibility that a pixel is in the foreground. The system discriminates foreground from background using similarity thresholding. We experimentally validated the proposed methodology for detecting pointing gestures.