A Model for Facial Expression Generation Using GAN for Improving Dialogue Quality of Agents

Abstract

We have been studying a model for generating facial expression videos that reflect the emotions of dialogue content in order to improve the human-like nature of dialogue agents. In a previous study, we proposed a model that can generate human-like facial expressions by learning the knowledge of lip-sync expressions and emotional facial expressions from different datasets. However, the generation results are inadequate due to the use of phonemes as input data and the frame rate of the generation results being too low. In this paper, we improve the model proposed in the previous study by using video as the input data and increasing the frame rate of the generated results to improve the quality of the results. In addition, by inputting the expression point video generated by the model into the facial expression video generation model for real images, we can generate facial images for emotional speech videos. For the facial expression video generation model, we use a model proposed by Zakharov et al. that can transfer facial expressions to arbitrary face images . The generated facial expression videos are subjected to sensitivity evaluation.