Automatic Hand Region Extraction and Finger Direction Recognition with a Depth Image Sensor for Japanese Sign Language Recognition

Abstract

Hand sign recognition is one of most challenging tasks in some research area. In this research, we focus on JSL (Japanese sign language). Although many hand sign recognition methods with depth sensor have already been proposed and they have achieved high accuracy, they have a problem such as the requirement that signer wears a color belt on the gesturing hand to extract a hand region precisely. Therefore we consider that there is still a room of improving the HRE (hand region extraction). In this paper, we propose a new automatic HRE method using a depth sensor and Time-Series Curve, which is one of contour features. Additionally, in the conventional SVM-based methods, brute-force matching of hand signs is applied for achieving high recognition performance. However, this process is required long processing time. To solve this problem, we focus on the FD (finger direction) of JSL. In this paper, we propose a new FD recognition method for effectively restricting the number of target signs in the recognition process. From the experimental results with 8 subjects, we have achieved the similar HRE accuracy compared with the conventional method without wearing a color belt and 96.5% of recognition rate of FD.