AlphaFace: High Fidelity and Real-time Face Swapper Robust to Facial Pose
This work addresses a specific challenge in face-swapping for applications requiring robustness to facial poses, representing an incremental improvement over existing methods.
The paper tackles the problem of face-swapping quality degradation under extreme facial poses by introducing AlphaFace, which uses a vision-language model with contrastive losses to achieve stronger identity representation and attribute preservation while maintaining real-time performance, surpassing state-of-the-art methods in pose-challenging cases.
Existing face-swapping methods often deliver competitive results in constrained settings but exhibit substantial quality degradation when handling extreme facial poses. To improve facial pose robustness, explicit geometric features are applied, but this approach remains problematic since it introduces additional dependencies and increases computational cost. Diffusion-based methods have achieved remarkable results; however, they are impractical for real-time processing. We introduce AlphaFace, which leverages an open-source vision-language model and CLIP image and text embeddings to apply novel visual and textual semantic contrastive losses. AlphaFace enables stronger identity representation and more precise attribute preservation, all while maintaining real-time performance. Comprehensive experiments across FF++, MPIE, and LPFF demonstrate that AlphaFace surpasses state-of-the-art methods in pose-challenging cases. The project is publicly available on `https://github.com/andrewyu90/Alphaface_Official.git'.