Content-Adaptive Motion Rate Adaption for Learned Video Compression
This work addresses video compression efficiency for applications requiring adaptive coding, but it is incremental as it builds on existing learned codecs.
The paper tackles the domain gap between training and test data in learned video compression by introducing an online motion rate adaptation scheme with a patch-level bit allocation map, showing improved rate-distortion performance on sequences with complicated motion.
This paper introduces an online motion rate adaptation scheme for learned video compression, with the aim of achieving content-adaptive coding on individual test sequences to mitigate the domain gap between training and test data. It features a patch-level bit allocation map, termed the $α$-map, to trade off between the bit rates for motion and inter-frame coding in a spatially-adaptive manner. We optimize the $α$-map through an online back-propagation scheme at inference time. Moreover, we incorporate a look-ahead mechanism to consider its impact on future frames. Extensive experimental results confirm that the proposed scheme, when integrated into a conditional learned video codec, is able to adapt motion bit rate effectively, showing much improved rate-distortion performance particularly on test sequences with complicated motion characteristics.