SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate Reduction
This work addresses the challenge of applying SR to compressed videos for video processing and compression industries, but it is incremental as it focuses on benchmarking and evaluation rather than introducing new methods.
The authors tackled the problem of super-resolution (SR) methods underperforming on compressed videos by developing a benchmark to analyze SR's capacity to upscale videos compressed with five standards, assessing 19 models and finding that some allow bitrate reduction without significant quality loss.
In recent years, there has been significant interest in Super-Resolution (SR), which focuses on generating a high-resolution image from a low-resolution input. Deep learning-based methods for super-resolution have been particularly popular and have shown impressive results on various benchmarks. However, research indicates that these methods may not perform as well on strongly compressed videos. We developed a super-resolution benchmark to analyze SR's capacity to upscale compressed videos. Our dataset employed video codecs based on five widely-used compression standards: H.264, H.265, H.266, AV1, and AVS3. We assessed 19 popular SR models using our benchmark and evaluated their ability to restore details and their susceptibility to compression artifacts. To get an accurate perceptual ranking of SR models, we conducted a crowd-sourced side-by-side comparison of their outputs. We found that some SR models, combined with compression, allow us to reduce the video bitrate without significant loss of quality. We also compared a range of image and video quality metrics with subjective scores to evaluate their accuracy on super-resolved compressed videos. The benchmark is publicly available at https://videoprocessing.ai/benchmarks/super-resolution-for-video-compression.html