High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning
This work addresses the problem of high-fidelity face swapping for applications in media and entertainment, representing an incremental improvement over existing diffusion-based methods.
The paper tackles the challenges of preserving identity and aligning attributes in diffusion-based face swapping by introducing an identity-constrained attribute-tuning framework with decoupled condition injection and post-training refinement. It achieves superior identity similarity and attribute consistency, setting a new state-of-the-art in high-fidelity face swapping.
Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression. Diffusion models, known for their superior generative capabilities, have recently shown promise in advancing face-swapping quality. This paper addresses two key challenges in diffusion-based face swapping: the prioritized preservation of identity over target attributes and the inherent conflict between identity and attribute conditioning. To tackle these issues, we introduce an identity-constrained attribute-tuning framework for face swapping that first ensures identity preservation and then fine-tunes for attribute alignment, achieved through a decoupled condition injection. We further enhance fidelity by incorporating identity and adversarial losses in a post-training refinement stage. Our proposed identity-constrained diffusion-based face-swapping model outperforms existing methods in both qualitative and quantitative evaluations, demonstrating superior identity similarity and attribute consistency, achieving a new state-of-the-art performance in high-fidelity face swapping.