Facial Attractiveness Factors and their Learning Processes by Comparing the Results of Class Activation Mapping-based Visualization Methods Using Convolutional Neural Networks

Abstract

Recently, computational studies of analysis of facial attractiveness features have attracted much attention because of the comprehensive understanding that is difficult to achieve using only experimental approaches. However, the differences in results between models and methods have not been examined in detail. In this study, a tuned convolutional neural network (CNN) model was constructed, and the results were confirmed using several methods to visualize features important for prediction. Results using gradient-weighted class activation mapping (Grad-CAM), Grad-CAM++, and Score-CAM methods showed that the eye area tended to be activated in highly attractive female images, consistent with findings in psychology. In contrast, some features showed different results depending on the method and training times. It was suggested that the model learns the highly universal and method-dependent features of facial attractiveness. This affective engineering approach contributes to understanding perceptual psychology and various engineering applications.