End-to-end Optimized Video Compression with MV-Residual Prediction
This work addresses video compression efficiency for applications like streaming, though it appears incremental as it builds on existing end-to-end methods.
The paper tackles P-frame video compression by proposing an end-to-end trainable framework with a joint MV-Residual prediction network, achieving the highest MS-SSIM on a validation set for a competition.
We present an end-to-end trainable framework for P-frame compression in this paper. A joint motion vector (MV) and residual prediction network MV-Residual is designed to extract the ensembled features of motion representations and residual information by treating the two successive frames as inputs. The prior probability of the latent representations is modeled by a hyperprior autoencoder and trained jointly with the MV-Residual network. Specially, the spatially-displaced convolution is applied for video frame prediction, in which a motion kernel for each pixel is learned to generate predicted pixel by applying the kernel at a displaced location in the source image. Finally, novel rate allocation and post-processing strategies are used to produce the final compressed bits, considering the bits constraint of the challenge. The experimental results on validation set show that the proposed optimized framework can generate the highest MS-SSIM for P-frame compression competition.