LVC-LGMC: Joint Local and Global Motion Compensation for Learned Video Compression
This work addresses a specific bottleneck in video compression for applications requiring efficient encoding, though it is incremental as it builds upon existing methods like DCVC-TCM.
The paper tackles the bottleneck of ignoring global contexts like large-scale motions in learned video compression by proposing a joint local and global motion compensation module (LGMC), which integrates flow nets with a linear-complexity cross-attention mechanism, resulting in significant rate-distortion performance improvements over the baseline DCVC-TCM.
Existing learned video compression models employ flow net or deformable convolutional networks (DCN) to estimate motion information. However, the limited receptive fields of flow net and DCN inherently direct their attentiveness towards the local contexts. Global contexts, such as large-scale motions and global correlations among frames are ignored, presenting a significant bottleneck for capturing accurate motions. To address this issue, we propose a joint local and global motion compensation module (LGMC) for leaned video coding. More specifically, we adopt flow net for local motion compensation. To capture global context, we employ the cross attention in feature domain for motion compensation. In addition, to avoid the quadratic complexity of vanilla cross attention, we divide the softmax operations in attention into two independent softmax operations, leading to linear complexity. To validate the effectiveness of our proposed LGMC, we integrate it with DCVC-TCM and obtain learned video compression with joint local and global motion compensation (LVC-LGMC). Extensive experiments demonstrate that our LVC-LGMC has significant rate-distortion performance improvements over baseline DCVC-TCM.