Preference prediction for images using facial expression in multiple image domains

Abstract

We are facing with ever-increasing amount of image contents including photos obtained by ourselves and images posted on SNS sites by others. In such a situation, it is essential to develop a technique that can recommend images preferred by a user without imposing much effort to the user. In this study, we conducted an experiment to obtain image preference data and developed a machine learning model that predicts image preference. In addition to the presented images, we also utilized recorded facial images as implicit information, and compared which features better predict image preference. Furthermore, we used two different image domains (lunchboxes and landscapes) to investigate how image domain influences the facial features useful for preference prediction. We showed that, in both domains, the performance of preference prediction improved significantly by incorporating facial features. By analyzing the contribution of facial features to model prediction, we also showed that facial features related to positive and negative emotions were important for lunchbox and landscape images, respectively. This suggests that human image preferences for different image domains are well predicted by a machine learning model, though the preference is manifested as distinct facial features across different image domains.