Grounding Degradations in Natural Language for All-In-One Video Restoration
This work addresses video restoration for various degradations without prior knowledge, though it appears incremental as it builds on foundation models.
The authors tackled the problem of all-in-one video restoration without requiring degradation knowledge by grounding degradation-aware semantic context in natural language via foundation models, achieving state-of-the-art performance on all proposed benchmarks.
In this work, we propose an all-in-one video restoration framework that grounds degradation-aware semantic context of video frames in natural language via foundation models, offering interpretable and flexible guidance. Unlike prior art, our method assumes no degradation knowledge in train or test time and learns an approximation to the grounded knowledge such that the foundation model can be safely disentangled during inference adding no extra cost. Further, we call for standardization of benchmarks in all-in-one video restoration, and propose two benchmarks in multi-degradation setting, three-task (3D) and four-task (4D), and two time-varying composite degradation benchmarks; one of the latter being our proposed dataset with varying snow intensity, simulating how weather degradations affect videos naturally. We compare our method with prior works and report state-of-the-art performance on all benchmarks.