Plug-and-Play Versatile Compressed Video Enhancement
This work addresses the challenge of maintaining video quality after compression for real-time cloud computing and downstream vision tasks, representing an incremental improvement by leveraging existing codec information.
The paper tackles the problem of visual quality degradation in compressed videos, which affects downstream vision models, by proposing a codec-aware enhancement framework that reuses compression information to adaptively enhance videos, achieving superior performance over existing methods and assisting multiple downstream tasks as a plug-and-play module.
As a widely adopted technique in data transmission, video compression effectively reduces the size of files, making it possible for real-time cloud computing. However, it comes at the cost of visual quality, posing challenges to the robustness of downstream vision models. In this work, we present a versatile codec-aware enhancement framework that reuses codec information to adaptively enhance videos under different compression settings, assisting various downstream vision tasks without introducing computation bottleneck. Specifically, the proposed codec-aware framework consists of a compression-aware adaptation (CAA) network that employs a hierarchical adaptation mechanism to estimate parameters of the frame-wise enhancement network, namely the bitstream-aware enhancement (BAE) network. The BAE network further leverages temporal and spatial priors embedded in the bitstream to effectively improve the quality of compressed input frames. Extensive experimental results demonstrate the superior quality enhancement performance of our framework over existing enhancement methods, as well as its versatility in assisting multiple downstream tasks on compressed videos as a plug-and-play module. Code and models are available at https://huimin-zeng.github.io/PnP-VCVE/.