CodeFormer++: Blind Face Restoration Using Deformable Registration and Deep Metric Learning
This work addresses the trade-off between visual quality and identity fidelity in blind face restoration, which is important for applications like photo enhancement and forensics, but it appears incremental as it builds on prior generative methods.
The paper tackles the problem of blind face restoration by proposing CodeFormer++, a framework that maximizes generative priors to achieve high-quality restoration while preserving identity, demonstrating superior performance in visual fidelity and identity consistency on real-world and synthetic datasets.
Blind face restoration (BFR) has attracted increasing attention with the rise of generative methods. Most existing approaches integrate generative priors into the restoration pro- cess, aiming to jointly address facial detail generation and identity preservation. However, these methods often suffer from a trade-off between visual quality and identity fidelity, leading to either identity distortion or suboptimal degradation removal. In this paper, we present CodeFormer++, a novel framework that maximizes the utility of generative priors for high-quality face restoration while preserving identity. We decompose BFR into three sub-tasks: (i) identity- preserving face restoration, (ii) high-quality face generation, and (iii) dynamic fusion of identity features with realistic texture details. Our method makes three key contributions: (1) a learning-based deformable face registration module that semantically aligns generated and restored faces; (2) a texture guided restoration network to dynamically extract and transfer the texture of generated face to boost the quality of identity-preserving restored face; and (3) the integration of deep metric learning for BFR with the generation of informative positive and hard negative samples to better fuse identity- preserving and generative features. Extensive experiments on real-world and synthetic datasets demonstrate that, the pro- posed CodeFormer++ achieves superior performance in terms of both visual fidelity and identity consistency.