Face Animation with an Attribute-Guided Diffusion Model
This work addresses the issue of low-quality animations for applications like video editing or virtual avatars, representing an incremental improvement over existing methods.
The paper tackles the problem of unnatural distortions and artifacts in face animation by proposing a Face Animation framework with an attribute-guided Diffusion Model (FADM), which improves photo-realistic talking-head generation and rectifies artifacts through iterative diffusion refinements, achieving superior performance on benchmarks.
Face animation has achieved much progress in computer vision. However, prevailing GAN-based methods suffer from unnatural distortions and artifacts due to sophisticated motion deformation. In this paper, we propose a Face Animation framework with an attribute-guided Diffusion Model (FADM), which is the first work to exploit the superior modeling capacity of diffusion models for photo-realistic talking-head generation. To mitigate the uncontrollable synthesis effect of the diffusion model, we design an Attribute-Guided Conditioning Network (AGCN) to adaptively combine the coarse animation features and 3D face reconstruction results, which can incorporate appearance and motion conditions into the diffusion process. These specific designs help FADM rectify unnatural artifacts and distortions, and also enrich high-fidelity facial details through iterative diffusion refinements with accurate animation attributes. FADM can flexibly and effectively improve existing animation videos. Extensive experiments on widely used talking-head benchmarks validate the effectiveness of FADM over prior arts.