Proposal of Recurrent Attention Module for Capturing Subtle Facial Expression Changes

Abstract

Facial expressions are expressed by subtle changes in the shape of the eyes and mouth, wrinkles between the eyebrows, etc. However, it is difficult to capture these changes. In this study, we propose a Recurrent Attention Module (RAM) and a facial expression recognition method using RAM to capture subtle changes in facial expressions. In our experiments, we used CK+ and eNTEFRACE05 databases. In CK+, the recognition accuracies of ConvLSTM and Enhanced ConvLSTM are 93.0% and 95.7%, respectively, while the addition of RAM improves the accuracies by 2.7% and 1.2%, respectively. Furthermore, the recognition accuracies of ConvLSTM and Enhanced ConvLSTM in the eNTERFACE05 database were 39.8% and 49.3%, respectively, while adding RAM improved the accuracies by 4.6% and 0.6%, respectively. In comparison with the conventional method, the proposed method could not outperform the conventional method in CK+. On the other hand, the proposed method improves the accuracy of the eNTERFACE05 database by 0.6% over the conventional method.