DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration
This work addresses flicker artifacts in video face restoration for applications in media and entertainment, representing a novel method for adapting image priors to video.
The paper tackled the challenge of maintaining temporal consistency in video face restoration by extending VQ-VAEs with Dirichlet-distributed continuous variables, achieving state-of-the-art performance on tasks like blind face restoration, video inpainting, and facial colorization.
Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at https://github.com/fudan-generative-vision/DicFace.