Assisting System in Touch-Panel Operation Using Stereo Camera

Abstract

In this paper, we propose a system that can be used to simplify touch-panel operation for people with visual disability. In the system, a user specifies the target button on the panel by a verbal input. The system detects the button and user's fingertip by analyzing images obtained through a stereo camera. The system helps in guiding a finger to the panel by indicating the direction of motion of the fingertip relative to the panel through sounds generated in a headphone. To devise an efficient navigation method, ways to indicate the direction of motion of the finger, different types of navigation sounds, and ways to indicate the distance between the finger and the panel were considered. The experiment showed the effectiveness of the proposed system and the average navigation time for the most efficient method was 9.7s.