FLAIR: A Conditional Diffusion Framework with Applications to Face Video Restoration
This work addresses the challenge of generating realistic and consistent face videos from low-quality inputs, which is important for applications in video enhancement and media, though it appears incremental by adapting diffusion models specifically for video.
The paper tackles the problem of preserving temporal coherence in face video restoration by introducing FLAIR, a conditional diffusion framework that outperforms state-of-the-art methods in tasks like super-resolution and deblurring on two high-quality datasets.
Face video restoration (FVR) is a challenging but important problem where one seeks to recover a perceptually realistic face videos from a low-quality input. While diffusion probabilistic models (DPMs) have been shown to achieve remarkable performance for face image restoration, they often fail to preserve temporally coherent, high-quality videos, compromising the fidelity of reconstructed faces. We present a new conditional diffusion framework called FLAIR for FVR. FLAIR ensures temporal consistency across frames in a computationally efficient fashion by converting a traditional image DPM into a video DPM. The proposed conversion uses a recurrent video refinement layer and a temporal self-attention at different scales. FLAIR also uses a conditional iterative refinement process to balance the perceptual and distortion quality during inference. This process consists of two key components: a data-consistency module that analytically ensures that the generated video precisely matches its degraded observation and a coarse-to-fine image enhancement module specifically for facial regions. Our extensive experiments show superiority of FLAIR over the current state-of-the-art (SOTA) for video super-resolution, deblurring, JPEG restoration, and space-time frame interpolation on two high-quality face video datasets.