Research on Method Study of Lip Reading Technology Using Machine Learning

Abstract

In order to solve the problem of voice recognition, which reduces the recognition rate in noisy places, we tried to realize a lip-reading technology that could recognize even without voice. Software dedicated to reading lips has been developed for English speakers, but it is considered difficult to apply it to Japanese, which has a small number of vowels. Therefore, in this study, we conducted an experiment targeting Japanese speakers. Data were collected by having 10 subjects speak 5 times for each of the 6 limited sentences. As a result of classifying the collected data by machine learning, the classification by k-Nearest Neighbor was the most accurate. However, at the same time, it became clear that the recognition accuracy differs depending on the subject. In the future, we would like to proceed with the development of recognition methods that take individual differences into consideration.