RelightVid: Temporal-Consistent Diffusion Model for Video Relighting
This addresses the challenge of applying diffusion models to video relighting for applications in video editing and visual effects, though it appears incremental as it builds on existing image relighting methods.
The paper tackles the problem of video relighting by introducing RelightVid, a diffusion-based framework that achieves arbitrary video relighting with high temporal consistency, preserving illumination priors without requiring intrinsic decomposition.
Diffusion models have demonstrated remarkable success in image generation and editing, with recent advancements enabling albedo-preserving image relighting. However, applying these models to video relighting remains challenging due to the lack of paired video relighting datasets and the high demands for output fidelity and temporal consistency, further complicated by the inherent randomness of diffusion models. To address these challenges, we introduce RelightVid, a flexible framework for video relighting that can accept background video, text prompts, or environment maps as relighting conditions. Trained on in-the-wild videos with carefully designed illumination augmentations and rendered videos under extreme dynamic lighting, RelightVid achieves arbitrary video relighting with high temporal consistency without intrinsic decomposition while preserving the illumination priors of its image backbone.