A Study of Emotion Estimation Model Using Unstructured Data

Abstract

In this paper, we aim to automatically create highlight videos of sports games based on emotion estimation from facial expressions. In doing so, we used a convolutional neural network, which is one of the emotion estimation methods from facial images. In addition, the emotion estimation focused on smiles, which are considered an important facial expression in highlight scenes. Models were created using images of the entire face and models using images of the mouth only, and the accuracy of the two types of smile discrimination models were compared. In addition, the discriminant basis of each model was visualized in a heat map.