Neural Video Compression with In-Loop Contextual Filtering and Out-of-Loop Reconstruction Enhancement
This work addresses video compression efficiency for applications like streaming and storage, presenting a systematic study that is incremental but with notable gains.
This paper tackles the problem of improving neural video compression by systematically studying enhancement filtering techniques, categorizing them into in-loop contextual filtering and out-of-loop reconstruction enhancement, and introducing an adaptive coding decision strategy. The result is a 7.71% reduction in bit rate compared to state-of-the-art neural video codecs.
This paper explores the application of enhancement filtering techniques in neural video compression. Specifically, we categorize these techniques into in-loop contextual filtering and out-of-loop reconstruction enhancement based on whether the enhanced representation affects the subsequent coding loop. In-loop contextual filtering refines the temporal context by mitigating error propagation during frame-by-frame encoding. However, its influence on both the current and subsequent frames poses challenges in adaptively applying filtering throughout the sequence. To address this, we introduce an adaptive coding decision strategy that dynamically determines filtering application during encoding. Additionally, out-of-loop reconstruction enhancement is employed to refine the quality of reconstructed frames, providing a simple yet effective improvement in coding efficiency. To the best of our knowledge, this work presents the first systematic study of enhancement filtering in the context of conditional-based neural video compression. Extensive experiments demonstrate a 7.71% reduction in bit rate compared to state-of-the-art neural video codecs, validating the effectiveness of the proposed approach.