FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping
This addresses the challenge of realistic face swapping in computer vision, with incremental improvements in pose preservation and identity fidelity.
The paper tackles the problem of high-fidelity face swapping by proposing FaceDancer, a single-stage method that improves identity transfer and better preserves pose and other attributes like occlusion and lighting, outperforming state-of-the-art networks in experiments.
In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identityn transfer, while having significantly better pose preservation than most of the previous methods.