DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration
This addresses the challenge of restoring faces from uncertain degradation patterns for applications in computer vision, though it appears incremental as it builds on existing diffusion models.
The paper tackles the problem of blind face restoration with low generalization across domains by proposing a Diffusion-Information-Diffusion framework, achieving high generalization and superior performance over state-of-the-art methods in real-world and heterogeneous settings.
Blind face restoration (BFR) is a highly challenging problem due to the uncertainty of degradation patterns. Current methods have low generalization across photorealistic and heterogeneous domains. In this paper, we propose a Diffusion-Information-Diffusion (DID) framework to tackle diffusion manifold hallucination correction (DiffMAC), which achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Specifically, the first diffusion stage aligns the restored face with spatial feature embedding of the low-quality face based on AdaIN, which synthesizes degradation-removal results but with uncontrollable artifacts for some hard cases. Based on Stage I, Stage II considers information compression using manifold information bottleneck (MIB) and finetunes the first diffusion model to improve facial fidelity. DiffMAC effectively fights against blind degradation patterns and synthesizes high-quality faces with attribute and identity consistencies. Experimental results demonstrate the superiority of DiffMAC over state-of-the-art methods, with a high degree of generalization in real-world and heterogeneous settings. The source code and models will be public.