Neural Virtual Try-On System considering 3D human model

Abstract

We propose a novel virtual try-on method based on Generative Adversarial Networks(GAN), which uses 3D surface model of body. In existing GAN-based methods(CAGAN, SwapGAN) sometimes do not work on a human image of rare posture. In our proposed method, by using DensePose to estimate a point corresponding to 3D surface model for each pixel point of 2D image, 3D surface based information is incorporated into our model. Therefore, it is possible to change clothes of people in various postures. Our proposed method uses a coase-to-fine strategy. First, {\it Parts Generation Network} generates parts and they are mapped to 2D image to produce coarse dressing image. After that, {\it Refine Network} refines the coarse dressing image. In our experiment, we show the result of the proposed method and our method has effect on rare postures by comparison with existing methods.