Face Reenactment with Diffusion Model and Its Application to Video Compression

Abstract

With the advancement of information technology, the use of images and videos has become common. However, the capacity of storagedevices and communication bandwidth is finite, so the demand forcompression has been increasing. In addition to conventional frequency-based compression, deeplearning-based compression methods such as generative adversarial networks (GAN) have been emerging in recentyears. According to the existing FaR-GAN, a face image with a certainfacial expression can be reconstructed from a reference face image of a person andthe coordinate data of 68 landmarks representing the facial expression, which can be used forefficient facial image compression. However, suchexisting methods have problems in terms of reconstruction accuracy and smoothnessbetween frames.In this study, we propose a method that reconstructs an image from theprevious frame using a diffusion model recurrently for smooth inter-framerepresentation while optimizing the trade-off between person identificationand facial expression generation in diffusion model-based face imagereconstruction.