Enhancing 2D Pose Estimation Models through Adversarial Learning Techniques

Abstract

2D pose estimation is utilized in sports and health analytics. Deep learning models have the potential to estimate poses using only a single human image without the need for motion capture suits. This study aimed to enhance the existing pose estimation model, PoseResNet, which uses Residual Nets to encode input images and to output heatmaps for relevant human joints. To improve this model, we employed the GAN method, training to generate realistic images through adversarial learning between a Generator and a Discriminator. For training 2D pose estimation, we used PoseResNet as the Generator and simple CNN layers implemented as the Discriminator. In our experiments, we employed the MPII Human Pose Dataset and compared three models: 1) PoseResNet, 2) PoseResNet employing adversarial learning based on Patch GAN, and 3) PoseResNet employing adversarial learning based on Patch GAN and CAM logits. Experimental results show that adapting PoseResNet to adversarial learning based on Patch GAN can lead to a significant improvement in the PCKh score, particularly when the adversarial loss is moderately scaled. However, we also observed that either using a strong scalar multiplication for adversarial loss or incorporating CAM logits tends to be less effective in enhancing the quality of pose estimation.