We introduce a fingertip touch recognition technique for general surfaces and a technique that estimates the direction of force on the basis of processed fingertip images captured by an overhead camera. When a finger presses on a hard surface, the distribution of color in the fingernail changes in accordance with the force applied. The first technique classifies the fingertip state (“press” or “non-press”) on the basis of the fingernail color information in the captured images. We use multiple EigenNail classifiers to achieve accurate classification of fingertip state under various lighting conditions. The second technique estimates the force direction by measuring the displacement of the fingertip. Through user studies, we demonstrated the feasibility of the proposed fingertip touch recognition techniques and were able to produce design guidelines for a pointing interface based on the proposed techniques.
