Butterfly: Multiple Reference Frames Feature Propagation Mechanism for Neural Video Compression
This work addresses the need for better low-latency video compression for applications like streaming, though it is incremental as it builds on existing neural video compression frameworks.
The paper tackles the problem of improving compression efficiency in neural video codecs by proposing a new multi-reference frame feature propagation mechanism called Butterfly, which achieves a 7.6% bitrate saving compared to a single-reference frame model on the HEVC Class D dataset.
Using more reference frames can significantly improve the compression efficiency in neural video compression. However, in low-latency scenarios, most existing neural video compression frameworks usually use the previous one frame as reference. Or a few frameworks which use the previous multiple frames as reference only adopt a simple multi-reference frames propagation mechanism. In this paper, we present a more reasonable multi-reference frames propagation mechanism for neural video compression, called butterfly multi-reference frame propagation mechanism (Butterfly), which allows a more effective feature fusion of multi-reference frames. By this, we can generate more accurate temporal context conditional prior for Contextual Coding Module. Besides, when the number of decoded frames does not meet the required number of reference frames, we duplicate the nearest reference frame to achieve the requirement, which is better than duplicating the furthest one. Experiment results show that our method can significantly outperform the previous state-of-the-art (SOTA), and our neural codec can achieve -7.6% bitrate save on HEVC Class D dataset when compares with our base single-reference frame model with the same compression configuration.