Kinectを用いたリアルタイム手話通訳システムの開発

抄録

Today, the number of hearing and language disabilities is about 360,000 and the number of sign language interpreters is 3,271. There are few sign language interpreters for hearing and language disabilities. The fewer interrupters lead to communication problem between the aurally challenged and normal hearer. Therefore, we developed a system for real time sign language interpreting using Kinect. In this study, we need to recognizing the arms movements and hand states using Kinect was needed. We estimated the arms movements from change of “WRIST” position, and hand states from change of “THUMB” position, “HANDTIP” position and “HANDSTATE”. The system recognized sign language movements from the combination of arms movement recognition method and hand states recognition method. We create the sign language database in advance. The computer collates acquiring data using Kinect with database and display a translation of coincident text on the monitor. We performed sign language movements in front of Kinect translate system. As the result, the developed system can recognize arms movement and hand state, and interpret easy sign language words.