Pose Guided Human Image Synthesis with Partially Decoupled GAN
This work addresses the challenge of generating realistic human images with pose changes for applications like virtual try-on or animation, but it is incremental as it builds on existing GAN-based approaches with specific improvements.
The paper tackles the problem of pose-guided human image synthesis by proposing a method that decouples the human body into parts and uses a multi-head attention module to preserve detailed texture and handle sharp pose deformations. The method outperforms existing state-of-the-art methods on Market-1501 and DeepFashion datasets in both qualitative and quantitative metrics.
Pose Guided Human Image Synthesis (PGHIS) is a challenging task of transforming a human image from the reference pose to a target pose while preserving its style. Most existing methods encode the texture of the whole reference human image into a latent space, and then utilize a decoder to synthesize the image texture of the target pose. However, it is difficult to recover the detailed texture of the whole human image. To alleviate this problem, we propose a method by decoupling the human body into several parts (\eg, hair, face, hands, feet, \etc) and then using each of these parts to guide the synthesis of a realistic image of the person, which preserves the detailed information of the generated images. In addition, we design a multi-head attention-based module for PGHIS. Because most convolutional neural network-based methods have difficulty in modeling long-range dependency due to the convolutional operation, the long-range modeling capability of attention mechanism is more suitable than convolutional neural networks for pose transfer task, especially for sharp pose deformation. Extensive experiments on Market-1501 and DeepFashion datasets reveal that our method almost outperforms other existing state-of-the-art methods in terms of both qualitative and quantitative metrics.