Neural Generation of Blocks for Video Coding
This work addresses incremental improvements in video coding efficiency for applications like streaming or storage, focusing on niche content scenarios.
The paper tackles the problem of improving video compression efficiency for specific content types like panning, zooming, or detailed scenes by using generative neural networks (GNNs) to generate blocks when they outperform traditional inter- and intra-prediction methods, resulting in lower rate-distortion.
Well-trained generative neural networks (GNN) are very efficient at compressing visual information for static images in their learned parameters but not as efficient as inter- and intra-prediction for most video content. However, for content entering a frame, such as during panning or zooming out, and content with curves, irregular shapes, or fine detail, generation by a GNN can give better compression efficiency (lower rate-distortion). This paper proposes encoding content-specific learned parameters of a GNN within a video bitstream at specific times and using the GNN to generate content for specific ranges of blocks and frames. The blocks to generate are just the ones for which generation gives more efficient compression than inter- or intra- prediction. This approach maximizes the usefulness of the information contained in the learned parameters.