Mental Fatigue Estimation Method by Lip Movement Using Machine Learning

Abstract

Sensing technology that can measure various human conditions easily and conveniently is currently being developed. Among information obtained from sensing technology, it is said that emotions and mental fatigue unconsciously appear, especially in facial information. In our research based on lip movement, we have confirmed that lip movement and smoothly utterance have been influenced due to daily physical condition and fatigue. We therefore considered that fatigue could be measured from the research that focused on lip movement. We considered change of the features of lip movements and estimated fatigue by machine learning using them. In this experiment, subjects performed a 90-min calculation as fatigue task, and lip movement and CFF were measured before and after the task. We confirm whether the subjects were fatigued from the calculation task using CFF, and the change of the opening area during utterance was compared before and after the task. As a result, it was confirmed that all subjects had a decrease in CFF value, and the opening lip area also showed a certain change before and after the task. Furthermore, we compared four machine learning methods based on the feature values of the lip movements in order to estimate fatigue from the measured lip movements. As a result of the fatigue estimation, SVM was able to get 89% highest accuracy with 5 features amount (open lip area, standard deviation of open lip area, utterance time, total of lip width, and height during utterance).